Focusing Mechanisms for Compressive Imaging Device

ABSTRACT

A compressive imaging (CI) device including a light modulator and a light sensor (e.g., a single-element sensor or a sensor array). The CI device may support the focusing of light on the light modulator and/or on the light sensor in a number of ways: (1) determining a focus indicator value by analyzing a 1D or 2D image in the incident light field; (2) measuring light spillover between modulator regions and light sensing elements—either at the level of voltage measurements or the level of reconstructed images; (3) measure noise in reconstructed sub-images; (4) measuring an amount high-frequency content in the incident light field; (5) incorporating a range finder to measure distance to the object being imaged; (6) incorporating an image sensor downstream from the modulator; and (7) splitting a portion of the incident light onto an image sensor, prior to the modulator.

RELATED APPLICATION DATA

This application claims the benefit of priority to U.S. ProvisionalApplication No. 61/372,826, filed on Aug. 11, 2010, entitled“Compressive Sensing Systems and Methods”, invented by Richard Baraniuk,Gary Woods, Kevin Kelly, Robert Bridge, Sujoy Chatterjee and LenoreMcMackin, which is hereby incorporated by reference in its entirety asthough fully and completely set forth herein.

FIELD OF THE INVENTION

The present invention relates to the field of compressive sensing, andmore particularly to systems and methods for focusing a compressiveimaging device.

DESCRIPTION OF THE RELATED ART

According to Nyquist theory, a signal x(t) whose signal energy issupported on the frequency interval [−B,B] may be reconstructed fromsamples {x(nT)} of the signal x(t), provided the rate f_(S)=1/T_(S) atwhich the samples are captured is sufficiently high, i.e., provided thatf_(s) is greater than 2B. Similarly, for a signal whose signal energy issupported on the frequency interval [A,B], the signal may bereconstructed from samples captured with sample rate greater than B-A. Afundamental problem with any attempt to capture a signal x(t) accordingto Nyquist theory is the large number of samples that are generated,especially when B (or B-A) is large. The large number of samples istaxing on memory resources and on the capacity of transmission channels.

Nyquist theory is not limited to functions of time. Indeed, Nyquisttheory applies more generally to any function of one or more realvariables. For example, Nyquist theory applies to functions of twospatial variables such as images, to functions of time and two spatialvariables such as video, and to the functions used in multispectralimaging, hyperspectral imaging, medical imaging and a wide variety ofother applications. In the case of an image I(x,y) that depends onspatial variables x and y, the image may be reconstructed from samplesof the image, provided the samples are captured with sufficiently highspatial density. For example, given samples {I(nΔx,mΔy)} captured alonga rectangular grid, the horizontal and vertical densities 1/Δx and 1/Δyshould be respectively greater than 2B_(x) and 2B_(y), where B_(x) andB_(y) are the highest x and y spatial frequencies occurring in the imageI(x,y). The same problem of overwhelming data volume is experienced whenattempting to capture an image according to Nyquist theory. The moderntheory of compressive sensing is directed to such problems.

Compressive sensing relies on the observation that many signals (e.g.,images or video sequences) of practical interest are not onlyband-limited but also sparse or approximately sparse when representedusing an appropriate choice of transformation, for example, atransformation such as a Fourier transform, a wavelet transform or adiscrete cosine transform (DCT). A signal vector v is said to beK-sparse with respect to a given transformation T when thetransformation of the signal vector, Tv, has no more than K non-zerocoefficients. A signal vector v is said to be sparse with respect to agiven transformation T when it is K-sparse with respect to thattransformation for some integer K much smaller than the number L ofcomponents in the transformation vector Tv.

A signal vector v is said to be approximately K-sparse with respect to agiven transformation T when the coefficients of the transformationvector, Tv, are dominated by the K largest coefficients (i.e., largestin the sense of magnitude or absolute value). In other words, if the Klargest coefficients account for a high percentage of the energy in theentire set of coefficients, then the signal vector v is approximatelyK-sparse with respect to transformation T. A signal vector v is said tobe approximately sparse with respect to a given transformation T when itis approximately K-sparse with respect to the transformation T for someinteger K much less than the number L of components in thetransformation vector Tv.

Given a sensing device that captures images with N samples per image andin conformity to the Nyquist condition on spatial rates, it is often thecase that there exists some transformation and some integer K very muchsmaller than N such that the transform of each captured image will beapproximately K sparse. The set of K dominant coefficients may vary fromone image to the next. Furthermore, the value of K and the selection ofthe transformation may vary from one context (e.g., imaging application)to the next. Examples of typical transforms that might work in differentcontexts are the Fourier transform, the wavelet transform, the DCT, theGabor transform, etc.

Compressive sensing specifies a way of operating on the N samples of animage so as to generate a much smaller set of samples from which the Nsamples may be reconstructed, given knowledge of the transform underwhich the image is sparse (or approximately sparse). In particular,compressive sensing invites one to think of the N samples as a vector vin an N-dimensional space and to imagine projecting the vector v ontoeach vector in a series of M vectors {R(i)} in the N-dimensional space,where M is larger than K but still much smaller than N. Each projectiongives a corresponding real number s(i), e.g., according to theexpression

s(i)=<v,R(i)>,

where the notation <v,R(i)> represents the inner product (or dotproduct) of the vector v and the vector R(i). Thus, the series of Mprojections gives a vector U including M real numbers. Compressivesensing theory further prescribes methods for reconstructing (orestimating) the vector v of N samples from the vector U of M realnumbers. For example, according to one method, one should determine thevector x that has the smallest length (in the sense of the L₁ norm)subject to the condition that ΦTx=U, where Φ is a matrix whose rows arethe transposes of the vectors R(i), where T is the transformation underwhich the image is K sparse or approximately K sparse.

Compressive sensing is important because, among other reasons, it allowsreconstruction of an image based on M measurements instead of the muchlarger number of measurements N recommended by Nyquist theory. Thus, forexample, a compressive sensing camera would be able to capture asignificantly larger number of images for a given size of image store,and/or, transmit a significantly larger number of images per unit timethrough a communication channel of given capacity.

As mentioned above, compressive sensing operates by projecting the imagevector v onto a series of M vectors. As discussed in U.S. patentapplication Ser. No. 11/379,688 (published as 2006/0239336 and inventedby Baraniuk et al.) and illustrated in FIG. 1, an imaging device (e.g.,camera) may be configured to take advantage of the compressive sensingparadigm by using a digital micromirror device (DMD) 40. An incidentlightfield 10 passes through a lens 20 and then interacts with the DMD40. The DMD includes a two-dimensional array of micromirrors, each ofwhich is configured to independently and controllably switch between twoorientation states. Each micromirror reflects a corresponding portion ofthe incident light field based on its instantaneous orientation. Anymicromirrors in a first of the two orientation states will reflect theircorresponding light portions so that they pass through lens 50. Anymicromirrors in a second of the two orientation states will reflecttheir corresponding light portions away from lens 50. Lens 50 serves toconcentrate the light portions from the micromirrors in the firstorientation state onto a photodiode (or photodetector) situated atlocation 60. Thus, the photodiode generates a signal whose amplitude atany given time represents a sum of the intensities of the light portionsfrom the micromirrors in the first orientation state.

The compressive sensing is implemented by driving the orientations ofthe micromirrors through a series of spatial patterns. Each spatialpattern specifies an orientation state for each of the micromirrors. Theoutput signal of the photodiode is digitized by an A/D converter 70. Inthis fashion, the imaging device is able to capture a series ofmeasurements {s(i)} that represent inner products (dot products) betweenthe incident light field and the series of spatial patterns withoutfirst acquiring the incident light field as a pixelized digital image.The incident light field corresponds to the vector v of the discussionabove, and the spatial patterns correspond to the vectors R(i) of thediscussion above. The measurements {s(i)} are used to algorithmicallyconstruct an image representing the incident light field.

The incident light field may be modeled by a function I(x,y,t) of twospatial variables and time. Assuming for the sake of discussion that theDMD comprises a rectangular array. The DMD implements a spatialmodulation of the incident light field so that the light field leavingthe DMD in the direction of the lens 50 might be modeled by

{I(nΔx,mΔy,t)*M(n,m,t)}

where m and n are integer indices, where I(nΔx,mΔy,t) represents theportion of the light field that is incident upon that (n,m)^(th) mirrorof the DMD at time t. The function M(n,m,t) represents the orientationof the (n,m)^(th) mirror of the DMD at time t. At sampling times, thefunction M(n,m,t) equals one or zero, depending on the state of thedigital control signal that controls the (n,m)^(th) mirror. Thecondition M(n,m,t)=1 corresponds to the orientation state that reflectsonto the path that leads to the lens 50. The condition M(n,m,t)=0corresponds to the orientation state that reflects away from the lens50.

The lens 50 concentrates the spatially-modulated light field

{I(nΔx,mΔy,t)*M(n,m,t)}

onto a light sensitive surface of the photodiode. Thus, the lens and thephotodiode together implement a spatial summation of the light portionsin the spatially-modulated light field:

${S(t)} = {\sum\limits_{n,m}{{I\left( {{n\; \Delta \; x},{m\; \Delta \; y},t} \right)}{{M\left( {n,m,t} \right)}.}}}$

Signal S(t) may be interpreted as the intensity at time t of theconcentrated spot of light impinging upon the light sensing surface ofthe photodiode. The A/D converter captures measurements of S(t). In thisfashion, the compressive sensing camera optically computes an innerproduct of the incident light field with each spatial pattern imposed onthe mirrors. The multiplication portion of the inner product isimplemented by the mirrors of the DMD. The summation portion of theinner product is implemented by the concentrating action of the lens andalso the integrating action of the photodiode.

In a compressive imaging (CI) device such as described above, theincident light field needs to be properly focused on the DMD or thequality of image reconstruction will suffer.

SUMMARY

A compressive imaging (CI) device includes a light modulator and a lightsensing device. The light modulator modulates an incident light streamwith a sequence of spatial patterns to obtain a modulated light stream.(The light modulator comprises an array of light modulating elements.)The light sensing device receives the modulated light stream andgenerates samples that represent intensity of the modulated light streamas a function of time. The samples may be used to reconstruct an image(or a sequence of images) representing the incident light stream.

The CI device may include an input optical subsystem (e.g., camera lens)situated in front of the light modulator to focus the incident lightstream onto the modulator. For quality image reconstruction, the imagecarried by the incident light stream should be in focus at the lightmodulator. This means properly adjusting the focus setting of the inputoptical subsystem and/or properly setting the optical distance betweenthe input optical subsystem and the light modulator. In someembodiments, the light modulator may be translatable, e.g., mounted on atranslation stage, which may be manually and/or electronicallycontrollable.

The CI device may also include post-modulation optics to concentrate orfocus or direct the modulated light stream onto the light sensingdevice. In some embodiments, the light sensing device includes (or is)an array of light sensing elements, in which case the post-modulationoptics may be used to focus the modulated light stream onto the lightsensing array. The light sensing elements are supposed to receivemodulated light from respective non-overlapping regions of the lightmodulator. When the light sensing array is properly positioned ororiented with respect to the modulated light stream and/or thepost-modulation optics, light is cleanly delivered from the regions ontotheir respective light sensing elements. However, when the light sensingarray is not properly positioned or oriented with respect to themodulated light stream and/or the post-modulation optics, light from thearbitrary region will spill over (cross over) to light sensing elementsother than the respective lights sensing element. In other words, thearbitrary light sensing element will receive light from regions otherthan the respective region. The spill over is large when the lightsensing array is far from its optimal position or orientation.Conversely, when the light sensing array is properly positioned andoriented, the spillover is small. Spillover adversely effects imagereconstruction quality, and thus, is a thing to be minimized. Thus, thesensor array may be translatable and/or rotatable, e.g., may be mountedon a translation stage, a rotation stage or a combinedtranslation-rotation stage, which may be manually and/or electronicallycontrollable.

This disclosure discusses a number of ways to achieve proper focus ofthe image onto the modulator and/or to achieve proper focus of themodulated light stream onto the light sensing array.

In one method, the sequence of spatial patterns is configured to effectthe movement of a region along a 1D path on the field of the lightmodulator. Thus, the corresponding samples from the light sensing devicerepresent a one-dimensional image. The 1D image may be analyzed usingany of a variety of techniques to derive a value indicating the extentto which the incident light is in focus at the modulator. The focussetting of the input optical subsystem and/or position of the lightmodulator may be translated until the focus indicator is optimized.(This method may be practiced even when the light sensing deviceincludes an array of light sensing elements, by summing the samples fromthe elements to obtain one sum value per spatial pattern. The 1D imageis then based on the sequence of sum values.) Alternatively, the 1Dimage may be displayed, allowing a user to control adjustments to focusbased on perceived quality of the 1D image.

In another method, the sequence of spatial patterns may be configured tosupport the reconstruction of images. Groups of the samples are used toreconstruct corresponding images. The images may be analyzed todetermine a focus indicator value for each image (or for selected onesof the images). Again, the focus-determining parameters of the CI devicemay be adjusted until the 2D focus metric is optimized. Alternatively,the reconstructed image may be displayed, allowing the user to controladjustments to focus based on perceived quality of the 1D image.

In another method, the sequence of spatial patterns may be configured toinclude a number of patterns with high spatial frequencies. Thecorresponding samples may then be used to measure the extent to whichthe image, as a pattern of light on the modulator, has high spatialfrequencies. The presence of high spatial frequencies is an indicator ofbeing in focus.

In another method, the CI device includes a range finder that determinesa range to the object being imaged, e.g., by transmitting a light signaland receiving the signal reflected by the object. In embodiments thatuse a digital micromirror device as the light modulator, the rangefinder may be configured to transmit and receive via mirrors in the offstate, while the light sensing device captures modulated light viamirrors in the on state. The range determined by the range finder may beused to set to focus-determining parameters of the CI device.

In another method, the samples acquired by a particular one of lightsensing elements may be used to algorithmically reconstruct an imagecovering the whole extent of the light modulator. Because the particularlight sensing element receives light from its respective region and alsospillover light from other regions, the reconstructed image includesimage information in the respective region and also in the other regions(especially adjacent regions). The sum of the image pixel values in theother regions (or adjacent regions) may be used as a measure ofspillover. The spillover estimation process may be performed for two ormore of the light sensing elements to determine a composite spillovervalue for the light sensing array. The position and/or orientation ofthe light sensing array may be adjusted until spillover (or compositespillover) is minimized.

In another method, the samples acquired by a particular one of the lightsensing elements may be used to algorithmically reconstruct a sub-imagerepresenting only the region of the light modulator belonging to theparticular light sensing element. In the reconstruction process, thedomain of the spatial patterns is restricted to that region. Thus,portions of modulated light that spilled over from the other regions tothe particular light sensing device have no way of being algorithmicallymapped to their correct spatial locations. The spillover thus expressesitself in the reconstructed sub-image as noise. Therefore, spillover maybe estimate by measuring the amount of noise present in the sub-image.This spillover estimation process may be performed for two or more ofthe light sensing elements, to determine corresponding spillover values,which may be averaged or maximized to determine a composite spillovervalue. The spillover value(s) or the composite spillover value may beused to drive adjustments to the position and/or orientation of thelight sensing array.

In another method, a spatial pattern may be asserted on the lightmodulator, where the spatial patterns is configured to turn off lightmodulating elements for the region corresponding a particular one of thelight sensing elements, but to turn on at least a subset of the lightmodulating elements in the other regions. Thus, the modulated lightarriving at the particular light sensing element represents spilloverlight from the other regions. The sample (voltage reading) from theparticular light sensing element may be used as an indicator aspillover. This spillover estimation process may be performed for two ormore of the light sensing elements, using two or more respective spatialpatterns, to obtain corresponding spillover values. The spillover valuesmay be combined to determined a composite spillover value. The spillovervalue(s) or the composite spillover value may be used to driveadjustments to the position and/or orientation of the light sensingarray.

In another method, the light modulator includes an array of mirrors.Portions of the incident light stream that are reflected by on-statemirrors are detected the light sensing device and used to performcompressive image acquisition. Portions of the incident light streamthat are reflected by on-state mirrors are detected by an image sensor.The images captured by the image sensor may be analyzed by determinefocus indicator values. The position of the image sensor may be adjusteduntil the focus indicator value is optimized, indicating proper focusfor modulated light on the image sensor. (Alternatively, the capturedimages may be displayed on a display screen. The user may then controladjustments to the position of the image sensor, to achieve optimalfocus as viewed on the display.) That optimizing position may be used todetermine an optimal position or translation for the light sensingdevice. The functional relationship between optimal positions of theimage sensor and optimal positions of the light sensing device may beknown from calibration, the results of which are stored in memory.

In another embodiment, the incident light stream is split (or separated)into two streams prior to the light modulator. The first stream isprovided to the light modulator; the second stream is provided to animage sensor. The images captured by the image sensor may be analyzed todetermine focus indicator values. The position of the image sensorand/or the focus setting of the input optical subsystem may be adjusteduntil the focus indicator value is optimized, indicating proper focusfor light on the image sensor. (Alternatively, the captured images maybe displayed on a display screen. The user may then make or specifyadjustments to the position of the image sensor and/or to the focussetting of the input optical subsystem, to achieve optimal focus asviewed on the display.) The optimizing position of the image sensor maybe used to determine an optimal position or translation for the lightsensing device. The functional relationship between optimal positions ofthe image sensor and optimal positions of the light sensing device maybe known from calibration. Similarly, the optimizing focus setting ofthe input optical subsystem may be used to determine an optimal positionor translation for the light sensing device. Again, the functionalrelationship between optimal focus settings of the input opticalsubsystem and optimal positions of the light sensing device may be knownfrom calibration.

Various additional embodiments are described in U.S. ProvisionalApplication No. 61/372,826, which is hereby incorporated by reference inits entirety as though fully and completely set forth herein.

BRIEF DESCRIPTION OF THE DRAWINGS

A better understanding of the present invention can be obtained when thefollowing detailed description of the preferred embodiments isconsidered in conjunction with the following drawings.

FIG. 1A illustrates a compressive sensing camera according to the priorart.

FIG. 1B illustrates a distance requirement for separation of an incidentlight beam and a modulated light beam from an digital micromirror device(DMD).

FIG. 2A illustrates one embodiment of a system 100 that is operable tocapture compressive imaging samples and also samples of background lightlevel. (LMU is an acronym for “light modulation unit”. MLS is an acronymfor “modulated light stream”. LSD is an acronym for “light sensingdevice”.)

FIG. 2B illustrates an embodiment of system 100 that includes aprocessing unit 150.

FIG. 2C illustrates an embodiment of system 100 that includes an opticalsubsystem 105 to focus received light L onto the light modulation unit110.

FIG. 2D illustrates an embodiment of system 100 that includes an opticalsubsystem 117 to direct or focus or concentrate the modulated lightstream MLS onto the light sensing device 130.

FIG. 2E illustrates an embodiment where the optical subsystem 117 isrealized by a lens 117L.

FIG. 2F illustrates an embodiment of system 100 that includes a controlunit that is configured to supply a series of spatial patterns to thelight modulation unit 110.

FIG. 2G illustrates an embodiment of a system 160 that includes aplurality of light sensing devices, each of which receives light from acorresponding spatial portion of the light modulation unit.

FIG. 2H illustrates an embodiment of a method for constructing an image,involving the construction of sub-image corresponding respectively thelight sensing devices of FIG. 2G.

FIG. 2I illustrates an embodiment of a system that uses four lightsensing devices to measure light from four respective quadrants of thelight modulation unit 110. The measurements from each light sensingdevice are used to construct a corresponding sub-image.

FIG. 3A illustrates system 200, where the light modulation unit 110 isrealized by a plurality of mirrors (collectively referenced by label110M).

FIG. 3B shows an embodiment of system 200 that includes the processingunit 150.

FIG. 4 shows an embodiment of system 200 that includes the opticalsubsystem 117 to direct or focus or concentrate the modulated lightstream MLS onto the light sensing device 130.

FIG. 5A shows an embodiment of system 200 where the optical subsystem117 is realized by the lens 117L.

FIG. 5B shows an embodiment of system 200 where the optical subsystem117 is realized by a mirror 117M and lens 117L in series.

FIG. 5C shows another embodiment of system 200 that includes a TIR prismpair 107.

FIG. 6 illustrates an embodiment of a method 600 for computingfocus-related information based on a one-dimensional path scan on thesurface of the light modulation unit.

FIG. 7 illustrates an embodiment of a method 700, also based on theone-dimensional path scan, but using an array of light sensing elements.

FIG. 8 illustrates an embodiment of a method 800 for changing a focussetting of a compressive imaging device based on samples acquired fromthe light sensing device.

FIG. 9 illustrates an embodiment of a method 900 that involvesdisplaying a one-dimensional image, where the image is based on thesamples acquired from the light sensing device.

FIG. 10 illustrates an embodiment of a method 1000 for computingfocus-related information based on algorithmically constructing animage.

FIG. 11 illustrates an embodiment of a method 1100, also involvingalgorithmic construction of an image, and use of an array of lightsensing elements.

FIG. 12 illustrates an embodiment of a method 1200, also involvingalgorithmic construction of an image.

FIG. 13 illustrates an embodiment of a method 1300, involving thedisplay of an algorithmically constructed image.

FIG. 14A illustrates an embodiment of a method 1400 for estimating lightspillover values based on samples acquired from a light sensing array.

FIG. 14B illustrates a set of patterns, according to one embodiment,that may be used to determine spillover values for a 2×2 array of lightsensing devices.

FIG. 15 illustrates an embodiment of a method 1500 for estimating lightspillover using a single light sensing element in an array of lightsensing elements.

FIG. 16A illustrates an embodiment of a method 1640 for computing afocus indicator based on the amount of noise present in algorithmicallyconstructed sub-images.

FIG. 16B illustrates an embodiment of a method 1660 for computing afocus indicator based on the amount of spillover present inalgorithmically constructed images.

FIG. 16C illustrates an example of an algorithmically constructed image,based on the samples from a single one of the light sensing elements.

FIG. 17 illustrates an embodiment of a method 1700 for computing a focusindicator based on the presentation of high-frequency spatial patternsof the light modulation unit.

FIG. 18 illustrates an embodiment of a system 1800 that include atime-of-flight determining device.

FIG. 19 illustrates one embodiment of the time-of-flight determiningdevice.

FIG. 20A illustrates an embodiment of system 1800, where mirrors areused to reflect the light streams S₁ and S₂ respectively.

FIG. 20B illustrates an embodiment of system 1800, where a dual TIRprism is used to separate the incident light stream and the reflectedlight streams S₁ and S₂.

FIG. 21 illustrates an embodiment of system 2100 including a rangefinder.

FIGS. 22 and 23 illustrate the use of a dual TIR prism in a compressiveimage device.

FIG. 24 illustrates an embodiment of a system 2400 that includes animage sensor to capture images of a light stream S₂ (off-state light).

FIG. 25 illustrates an embodiment of system 2400 where mirrors 117M and118M are used to reflect the light streams S₁ and S₂.

FIG. 26 illustrates an embodiment of system 2400 using a dual TIR prism.

FIG. 27A illustrates an embodiment of a system 2700 that splits theincident light stream into two streams prior to modulation; the lightmodulator receives one of the streams; the image sensor receives theother stream.

FIG. 27B illustrates an embodiment of system 2700 where images from theimage sensor are displayed.

FIG. 28C illustrates an embodiment of system 2700 where the lightstreams S₁ and S₂ are produced by a TIR prism 2710.

FIG. 28D illustrates an embodiment where the TIR prism pair includes adielectric coating at an interface of the first prism.

FIGS. 28 and 29 shows embodiment of a compressive imaging (CI) systemhaving multiple signal acquisition channels.

While the invention is susceptible to various modifications andalternative forms, specific embodiments thereof are shown by way ofexample in the drawings and are herein described in detail. It should beunderstood, however, that the drawings and detailed description theretoare not intended to limit the invention to the particular formdisclosed, but on the contrary, the intention is to cover allmodifications, equivalents and alternatives falling within the spiritand scope of the present invention as defined by the appended claims.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The following patent applications provide teachings regardingcompressive imaging, and each of them is hereby incorporated byreference in its entirety as though fully and completely set forthherein

U.S. Provisional Application No. 61/372,826, filed on Aug. 11, 2010,entitled “Compressive Sensing Systems and Methods”, invented by RichardBaraniuk, Gary Woods, Kevin Kelly, Robert Bridge, Sujoy Chatterjee andLenore McMackin.

U.S. Application No. 13/193,553, filed on Jul. 28, 2011, entitled“Determining Light Level Variation in Compressive Imaging by InjectingCalibration Patterns into Pattern Sequence”, invented by RichardBaraniuk, Kevin Kelly, Robert Bridge, Sujoy Chatterjee and LenoreMcMackin.

U.S. Application No. 13/193,556, filed on Jul. 28, 2011, entitled“Low-Pass Filtering of Compressive Imaging Measurements to Infer LightLevel Variation”, invented by Richard Baraniuk, Kevin Kelly, RobertBridge, Sujoy Chatterjee and Lenore McMackin.

U.S. Application No. 13/197,304, filed on Aug. 3, 2011, entitled“Decreasing Image Acquisition Time for Compressive Imaging Devices”,invented by Woods et al.

U.S. Application No. 13/207,276, filed on Aug. 10, 2011, entitled“Dynamic Range Optimization and Hot Spot Correction in a CompressiveImaging System”, invented by Gary Woods and James Tidman.

U.S. Application No. 13/207,258, filed on Aug. 10, 2011, entitled“Techniques for Removing Noise in a Compressive Imaging Device”,invented by Woods et al.

U.S. Application No. 13/207,900, filed on Aug. 11, 2011, entitled “TIRPrism to Separate Incident Light and Modulated Light in CompressiveImaging Device”, invented by Lenore McMackin and Sujoy Chatterjee.

Terminology

A memory medium is a non-transitory medium configured for the storageand retrieval of information. Examples of memory media include: variouskinds of semiconductor-based memory such as RAM and ROM; various kindsof magnetic media such as magnetic disk, tape, strip and film; variouskinds of optical media such as CD-ROM and DVD-ROM; various media basedon the storage of electrical charge and/or any of a wide variety ofother physical quantities; media fabricated using various lithographictechniques; etc. The term “memory medium” includes within its scope ofmeaning the possibility that a given memory medium might be a union oftwo or more memory media that reside at different locations, e.g., ondifferent chips in a system or on different computers in a network. Amemory medium is typically computer-readable, e.g., is capable of beingread by a computer.

A computer-readable memory medium may be configured so that it storesprogram instructions and/or data, where the program instructions, ifexecuted by a computer system, cause the computer system to perform amethod, e.g., any of a method embodiments described herein, or, anycombination of the method embodiments described herein, or, any subsetof any of the method embodiments described herein, or, any combinationof such subsets.

A computer system is any device (or combination of devices) having atleast one processor that is configured to execute program instructionsstored on a memory medium. Examples of computer systems include personalcomputers (PCs), workstations, laptop computers, tablet computers,mainframe computers, server computers, client computers, network orInternet appliances, hand-held devices, mobile devices, personal digitalassistants (PDAs), computer-based television systems, grid computingsystems, wearable computers, computers implanted in living organisms,computers embedded in head-mounted displays, computers embedded insensors forming a distributed network, etc.

A programmable hardware element (PHE) is a hardware device that includesmultiple programmable function blocks connected via a system ofprogrammable interconnects. Examples of PHEs include FPGAs (FieldProgrammable Gate Arrays), PLDs (Programmable Logic Devices), FPOAs(Field Programmable Object Arrays), and CPLDs (Complex PLDs). Theprogrammable function blocks may range from fine grained (combinatoriallogic or look up tables) to coarse grained (arithmetic logic units orprocessor cores).

As used herein, the term “light” is meant to encompass within its scopeof meaning any electromagnetic radiation whose spectrum lies within thewavelength range [λ_(L), λ_(U)], where the wavelength range includes thevisible spectrum, the ultra-violet (UV) spectrum, infrared (IR) spectrumand the terahertz (THz) spectrum. Thus, for example, visible radiation,or UV radiation, or IR radiation, or THz radiation, or any combinationthereof is “light” as used herein.

In some embodiments, a computer system may be configured to include aprocessor (or a set of processors) and a memory medium, where the memorymedium stores program instructions, where the processor is configured toread and execute the program instructions stored in the memory medium,where the program instructions are executable by the processor toimplement a method, e.g., any of the various method embodimentsdescribed herein, or, any combination of the method embodimentsdescribed herein, or, any subset of any of the method embodimentsdescribed herein, or, any combination of such subsets.

System 100 for Operating on Light

A system 100 for operating on light may be configured as shown in FIG.2A. The system 100 may include a light modulation unit 110, a lightsensing device 130 and an analog-to-digital converter (ADC) 140.

The light modulation unit 110 is configured to modulate a receivedstream of light L with a series of spatial patterns in order to producea modulated light stream (MLS). The spatial patterns of the series maybe applied sequentially to the light stream so that successive timeslices of the light stream are modulated, respectively, with successiveones of the spatial patterns. (The action of sequentially modulating thelight stream L with the spatial patterns imposes the structure of timeslices on the light stream.) The light modulation unit 110 includes aplurality of light modulating elements configured to modulatecorresponding portions of the light stream. Each of the spatial patternsspecifies an amount (or extent or value) of modulation for each of thelight modulating elements. Mathematically, one might think of the lightmodulation unit's action of applying a given spatial pattern asperforming an element-wise multiplication of a light field vector(x_(1j)) representing a time slice of the light stream L by a vector ofscalar modulation values (m_(1j)) to obtain a time slice of themodulated light stream: (m_(1j))*(x_(1j))=(m_(1j)+x_(1j)). The vector(m_(1j)) is specified by the spatial pattern. Each light modulatingelement effectively scales (multiplies) the intensity of itscorresponding stream portion by the corresponding scalar factor.

The light modulation unit 110 may be realized in various ways. In someembodiments, the LMU 110 may be realized by a plurality of mirrors(e.g., micromirrors) whose orientations are independently controllable.In another set of embodiments, the LMU 110 may be realized by an arrayof elements whose transmittances are independently controllable, e.g.,as with an array of LCD shutters. An electrical control signal suppliedto each element controls the extent to which light is able to transmitthrough the element. In yet another set of embodiments, the LMU 110 maybe realized by an array of independently-controllable mechanicalshutters (e.g., micromechanical shutters) that cover an array ofapertures, with the shutters opening and closing in response toelectrical control signals, thereby controlling the flow of lightthrough the corresponding apertures. In yet another set of embodiments,the LMU 110 may be realized by a perforated mechanical plate, with theentire plate moving in response to electrical control signals, therebycontrolling the flow of light through the corresponding perforations. Inyet another set of embodiments, the LMU 110 may be realized by an arrayof transceiver elements, where each element receives and thenretransmits light in a controllable fashion. In yet another set ofembodiments, the LMU 110 may be realized by a grating light valve (GLV)device. In yet another embodiment, the LMU 110 may be realized by aliquid-crystal-on-silicon (LCOS) device.

In some embodiments, the light modulating elements are arranged in anarray, e.g., a two-dimensional array or a one-dimensional array. Any ofvarious array geometries are contemplated. For example, in someembodiments, the array is a square array or rectangular array. Inanother embodiment, the array is hexagonal. In some embodiments, thelight modulating elements are arranged in a spatially random fashion.

Let N denote the number of light modulating elements in the lightmodulation unit 110. In various embodiments, the number N may take awide variety of values. For example, in different sets of embodiments, Nmay be, respectively, in the range [64, 256], in the range [256, 1024],in the range [1024, 4096], in the range [2¹²,2¹⁴], in the range[2¹⁴,2¹⁶], in the range [2¹⁶,2¹⁸], in the range [2¹⁸,2²⁰], in the range[2²⁰,2²²], in the range [2²²,2²⁴], in the range [2²⁴,2²⁶], in the rangefrom 2²⁶ to infinity. The particular value used in any given embodimentmay depend on one or more factors specific to the embodiment.

The light sensing device 130 is configured to receive the modulatedlight stream MLS and to generate an analog electrical signal I_(MLS)(t)representing intensity of the modulated light stream as a function oftime.

The light sensing device 130 may include one or more light sensingelements. The term “light sensing element” may be interpreted as meaning“a transducer between a light signal and an electrical signal”. Forexample, a photodiode is a light sensing element. In various otherembodiments, light sensing elements might include devices such asmetal-semiconductor-metal (MSM) photodetectors, phototransistors,phototubes and photomultiplier tubes.

In some embodiments, the light sensing device 130 includes one or moreamplifiers (e.g., transimpedance amplifiers) to amplify the analogelectrical signals generated by the one or more light sensing elements.

The ADC 140 acquires a sequence of samples {I_(MLS)(k)} of the analogelectrical signal I_(MLS)(t). Each of the samples may be interpreted asan inner product between a corresponding time slice of the light streamL and a corresponding one of the spatial patterns. The set of samples{I_(MLS)(k)} comprises an encoded representation, e.g., a compressedrepresentation, of an image (or a video sequence) and may be used toconstruct the image (or video sequence) based on any constructionalgorithm known in the field of compressive sensing. (For video sequenceconstruction, the samples may be partitioned into contiguous subsets,and then the subsets may be processed to construct correspondingimages.)

In some embodiments, the samples {I_(MLS)(k)} may be used for somepurpose other than, or in addition to, image (or video) construction.For example, system 100 (or some other system) may operate on thecompensated samples to perform an inference task, such as detecting thepresence of a signal or object, identifying a signal or an object,classifying a signal or an object, estimating one or more parametersrelating to a signal or an object, tracking a signal or an object, etc.In some embodiments, an object under observation by system 100 may beidentified or classified by virtue of its sample set {I_(MLS)(k)}, orparameters derived from that sample set, being similar to one of acollection of stored sample sets (or parameter sets).

In some embodiments, the light sensing device 130 includes exactly onelight sensing element. (For example, the single light sensing elementmay be a photodiode.) The light sensing element may couple to anamplifier (e.g., a TIA) (e.g., a multi-stage amplifier).

In some embodiments, the light sensing device 130 may include aplurality of light sensing elements (e.g., photodiodes). Each lightsensing element may convert light impinging on its light sensing surfaceinto a corresponding analog electrical signal representing intensity ofthe impinging light as a function of time. In some embodiments, eachlight sensing element may couple to a corresponding amplifier so thatthe analog electrical signal produced by the light sensing element canbe amplified prior to digitization. System 100 may be configured so thateach light sensing element receives, e.g., a corresponding spatialportion (or spectral portion) of the modulated light stream.

In one embodiment, the analog electrical signals produced, respectively,by the light sensing elements may be summed to obtain a sum signal. Thesum signal may then be digitized by the ADC 140 to obtain the sequenceof samples {I_(MLS)(k)}. In another embodiment, the analog electricalsignals may be individually digitized, each with its own ADC, to obtaincorresponding sample sequences. The sample sequences may then be addedto obtain the sequence {I_(MLS)(k)}. In another embodiment, the analogelectrical signals produced by the light sensing elements may be sampledby a smaller number of ADCs than light sensing elements through the useof time multiplexing. For example, in one embodiment, system 100 may beconfigured to sample two or more of the analog electrical signals byswitching the input of an ADC among the outputs of the two or morecorresponding light sensing elements at a sufficiently high rate.

In some embodiments, the light sensing device 130 may include an arrayof light sensing elements. Arrays of any of a wide variety of sizes,configurations and material technologies are contemplated. In oneembodiment, the light sensing device 130 includes a focal plane arraycoupled to a readout integrated circuit. In one embodiment, the lightsensing device 130 may include an array of cells, where each cellincludes a corresponding light sensing element and is configured tointegrate and hold photo-induced charge created by the light sensingelement, and to convert the integrated charge into a corresponding cellvoltage. The light sensing device may also include (or couple to)circuitry configured to sample the cell voltages using one or more ADCs.

In some embodiments, the light sensing device 130 may include aplurality (or array) of light sensing elements, where each light sensingelement is configured to receive a corresponding spatial portion of themodulated light stream, and each spatial portion of the modulated lightstream comes from a corresponding sub-region of the array of lightmodulating elements. (For example, the light sensing device 130 mayinclude a quadrant photodiode, where each quadrant of the photodiode isconfigured to receive modulated light from a corresponding quadrant ofthe array of light modulating elements. As another example, the lightsensing element 130 may include a bi-cell photodiode.) Each lightsensing element generates a corresponding signal representing intensityof the corresponding spatial portion as a function of time. Each signalmay be digitized (e.g., by a corresponding ADC) to obtain acorresponding sequence of samples. Each sequence of samples may beprocessed to recover a corresponding sub-image. The sub-images may bejoined together to form a whole image.

In some embodiments, the light sensing device 130 includes a smallnumber of light sensing elements (e.g., in respective embodiments, one,two, less than 8, less than 16, less the 32, less than 64, less than128, less than 256). Because the light sensing device of theseembodiments includes a small number of light sensing elements (e.g., farless than the typical modern CCD-based or CMOS-based camera), an entityconfiguring any of these embodiments may afford to spend more per lightsensing element to realize features that are beyond the capabilities ofmodern array-based image sensors of large pixel count, e.g., featuressuch as higher sensitivity, extended range of sensitivity, new range(s)of sensitivity, extended dynamic range, higher bandwidth/lower responsetime. Furthermore, because the light sensing device includes a smallnumber of light sensing elements, an entity configuring any of theseembodiments may use newer light sensing technologies (e.g., based on newmaterials or combinations of materials) that are not yet mature enoughto be manufactured into focal plane arrays (FPA) with large pixel count.For example, new detector materials such as super-lattices, quantumdots, carbon nanotubes and graphene can significantly enhance theperformance of IR detectors by reducing detector noise, increasingsensitivity, and/or decreasing detector cooling requirements.

In one embodiment, the light sensing device 130 is a thermo-electricallycooled InGaAs detector. (InGaAs stands for “Indium Gallium Arsenide”.)In other embodiments, the InGaAs detector may be cooled by othermechanisms (e.g., liquid nitrogen or a Sterling engine). In yet otherembodiments, the InGaAs detector may operate without cooling. In yetother embodiments, different detector materials may be used, e.g.,materials such as MCT (mercury-cadmium-telluride), InSb (IndiumAntimonide) and VOx (Vanadium Oxide).

In different embodiments, the light sensing device 130 may be sensitiveto light at different wavelengths or wavelength ranges. In someembodiments, the light sensing device 130 may be sensitive to light overa broad range of wavelengths, e.g., over the entire visible spectrum orover the entire range [λ_(L),λX_(U)] as defined above.

In some embodiments, the light sensing device 130 may include one ormore dual-sandwich photodetectors. A dual sandwich photodetectorincludes two photodiodes stacked (or layered) one on top of the other.

In one embodiment, the light sensing device 130 may include one or moreavalanche photodiodes.

In some embodiments, a filter may be placed in front of the lightsensing device 130 to restrict the modulated light stream to a specificrange of wavelengths or polarization. Thus, the signal I_(MLS)(t)generated by the light sensing device 130 may be representative of theintensity of the restricted light stream. For example, by using a filterthat passes only IR light, the light sensing device may be effectivelyconverted into an IR detector. The sample principle may be applied toeffectively convert the light sensing device into a detector for red orblue or green or UV or any desired wavelength band, or, a detector forlight of a certain polarization.

In some embodiments, system 100 includes a color wheel whose rotation issynchronized with the application of the spatial patterns to the lightmodulation unit. As it rotates, the color wheel cyclically applies anumber of optical bandpass filters to the modulated light stream MLS.Each bandpass filter restricts the modulated light stream to acorresponding sub-band of wavelengths. Thus, the samples captured by theADC 140 will include samples of intensity in each of the sub-bands. Thesamples may be de-multiplexed to form separate sub-band sequences. Eachsub-band sequence may be processed to generate a corresponding sub-bandimage. (As an example, the color wheel may include a red-pass filter, agreen-pass filter and a blue-pass filter to support color imaging.)

In some embodiments, the system 100 may include a memory (or a set ofmemories of one or more kinds).

In some embodiments, system 100 may include a processing unit 150, e.g.,as shown in FIG. 2B. The processing unit 150 may be a digital circuit ora combination of digital circuits. For example, the processing unit maybe a microprocessor (or system of interconnected of microprocessors), aprogrammable hardware element such as a field-programmable gate array(FPGA), an application specific integrated circuit (ASIC), or anycombination such elements. The processing unit 150 may be configured toperform one or more functions such as image/video construction, systemcontrol, user interface, statistical analysis, and one or moreinferences tasks. In some embodiments, any subset of the variouscomputational operations described herein may be performed by processingunit 150.

The system 100 (e.g., the processing unit 150) may store the samples{I_(MLS)(k)} in a memory, e.g., a memory resident in the system 100 orin some other system.

In one embodiment, processing unit 150 is configured to operate on thesamples {I_(MLS)(k)} to generate the image or video sequence. In thisembodiment, the processing unit 150 may include a microprocessorconfigured to execute software (i.e., program instructions), especiallysoftware for performing an image/video construction algorithm. In oneembodiment, system 100 is configured to transmit the compensated samplesto some other system through a communication channel. (In embodimentswhere the spatial patterns are randomly-generated, system 100 may alsotransmit the random seed(s) used to generate the spatial patterns.) Thatother system may operate on the samples to construct the image/video.System 100 may have one or more interfaces configured for sending (andperhaps also receiving) data through one or more communication channels,e.g., channels such as wireless channels, wired channels, fiber opticchannels, acoustic channels, laser-based channels, etc.

In some embodiments, processing unit 150 is configured to use any of avariety of algorithms and/or any of a variety of transformations toperform image/video construction. System 100 may allow a user to choosea desired algorithm and/or a desired transformation for performing theimage/video construction.

In some embodiments, the system 100 is configured to acquire a set Z_(M)of samples from the ADC 140 so that the sample set Z_(M) corresponds toM of the spatial patterns applied to the light modulation unit 110,where M is a positive integer. The number M is selected so that thesample set Z_(M) is useable to construct an n-pixel image or n-voxelvideo sequence that represents the incident light stream, where n is apositive integer less than or equal to the number N of light modulatingelements in the light modulation unit 110. System 100 may be configuredso that the number M is smaller than n. Thus, system 100 may operate asa compressive sensing device. (The number of “voxels” in a videosequence is the number of images in the video sequence times the numberof pixels per image, or equivalently, the sum of the pixel counts of theimages in the video sequence.)

In various embodiments, the compression ratio M/n may take any of a widevariety of values. For example, in different sets of embodiments, M/nmay be, respectively, in the range [0.9,0.8], in the range [0.8,0.7], inthe range [0.7,0.6], in the range [0.6,0.5], in the range [0.5,0.4], inthe range [0.4,0.3], in the range [0.3,0.2], in the range [0.2,0.1], inthe range [0.1,0.05], in the range [0.05,0.01], in the range[0.001,0.01].

As noted above, the image constructed from the sample subset Z_(M) maybe an n-pixel image with n≦N. The spatial patterns may be designed tosupport a value of n less than N, e.g., by forcing the array of lightmodulating elements to operate at a lower effective resolution than thephysical resolution N. For example, the spatial patterns may be designedto force each 2×2 cell of light modulating elements to act in unison. Atany given time, the modulation state of the four elements in a 2×2 cellwill agree. Thus, the effective resolution of the array of lightmodulating elements is reduced to N/4. This principle generalizes to anycell size, to cells of any shape, and to collections of cells withnon-uniform cell size and/or cell shape. For example, a collection ofcells of size k_(H)×k_(v), where k_(H) and k_(v) are positive integers,would give an effective resolution equal to N/(k_(H)k_(v)). In onealternative embodiment, cells near the center of the array may havesmaller sizes than cells near the periphery of the array.

Another way the spatial patterns may be arranged to support theconstruction of an n-pixel image with n less than N is to allow thespatial patterns to vary only within a subset of the array of lightmodulating elements. In this mode of operation, the spatial patterns arenull (take the value zero) outside the subset. (Control unit 120 may beconfigured to implement this restriction of the spatial patterns.) Thus,light modulating elements corresponding to positions outside of thesubset do not send any light (or send only the minimum amount of lightattainable) to the light sensing device. Thus, the constructed image isrestricted to the subset. In some embodiments, each spatial pattern(e.g., of a measurement pattern sequence) may be multiplied element-wiseby a binary mask that takes the one value only in the allowed subset,and the resulting product pattern may be supplied to the lightmodulation unit. In some embodiments, the subset is a contiguous regionof the array of light modulating elements, e.g., a rectangle or acircular disk or a hexagon. In some embodiments, the size and/orposition of the region may vary (e.g., dynamically). The position of theregion may vary in order to track a moving object. The size of theregion may vary to dynamically control the rate of image acquisition.

In one embodiment, system 100 may include a light transmitter configuredto generate a light beam

(e.g., a laser beam), to modulate the light beam with a data signal andto transmit the modulated light beam into space or onto an opticalfiber. System 100 may also include a light receiver configured toreceive a modulated light beam from space or from an optical fiber, andto recover a data stream from the received modulated light beam.

In one embodiment, system 100 may be configured as a low-cost sensorsystem having minimal processing resources, e.g., processing resourcesinsufficient to perform image (or video) construction in user-acceptabletime. In this embodiment, the system 100 may store and/or transmit thesamples {I_(MLS)(k)} so that another agent, more plentifully endowedwith processing resources, may perform the image/video constructionbased on the samples.

In some embodiments, system 100 may include an optical subsystem 105that is configured to modify or condition or operate on the light streamL before it arrives at the light modulation unit 110, e.g., as shown inFIG. 2C. For example, the optical subsystem 105 may be configured toreceive the light stream L from the environment and to focus the lightstream onto a modulating plane of the light modulation unit 110. Theoptical subsystem 105 may include a camera lens (or a set of lenses).The lens (or set of lenses) may be adjustable to accommodate a range ofdistances to external objects being imaged/sensed/captured. In someembodiments, the optical subsystem 105 includes a conventional cameralens unit, e.g., configured for mounting in a standard mount.

In some embodiments, system 100 may include an optical subsystem 117 todirect the modulated light stream MLS onto a light sensing surface (orsurfaces) of the light sensing device 130.

In some embodiments, the optical subsystem 117 may include one or morelenses, and/or, one or more mirrors.

In some embodiments, the optical subsystem 117 is configured to focusthe modulated light stream onto the light sensing surface (or surfaces).The term “focus” implies an attempt to achieve the condition that rays(photons) diverging from a point on an object plane converge to a point(or an acceptably small spot) on an image plane. The term “focus” alsotypically implies continuity between the object plane point and theimage plane point (or image plane spot)—points close together on theobject plane map respectively to points (or spots) close together on theimage plane. In at least some of the system embodiments that include anarray of light sensing elements, it may be desirable for the modulatedlight stream MLS to be focused onto the light sensing array so thatthere is continuity between points on the light modulation unit LMU andpoints (or spots) on the light sensing array.

In some embodiments, the optical subsystem 117 may be configured todirect the modulated light stream MLS onto the light sensing surface (orsurfaces) of the light sensing device 130 in a non-focusing fashion. Forexample, in a system embodiment that includes only one photodiode, itmay not be so important to achieve the “in focus” condition at the lightsensing surface of the photodiode since positional information ofphotons arriving at that light sensing surface will be immediately lost.

In one embodiment, the optical subsystem 117 may be configured toreceive the modulated light stream and to concentrate the modulatedlight stream into an area (e.g., a small area) on a light sensingsurface of the light sensing device 130. Thus, the diameter of themodulated light stream may be reduced (possibly, radically reduced) inits transit from the optical subsystem 117 to the light sensing surface(or surfaces) of the light sensing device 130. For example, in someembodiments, the diameter may be reduced by a factor of more than 1.5to 1. In other embodiments, the diameter may be reduced by a factor ofmore than 2 to 1. In yet other embodiments, the diameter may be reducedby a factor of more than 10 to 1. In yet other embodiments, the diametermay be reduced by factor of more than 100 to 1. In yet otherembodiments, the diameter may be reduced by factor of more than 400to 1. In one embodiment, the diameter is reduced so that the modulatedlight stream is concentrated onto the light sensing surface of a singlelight sensing element (e.g., a single photodiode).

In some embodiments, this feature of concentrating the modulated lightstream onto the light sensing surface (or surfaces) of the light sensingdevice allows the light sensing device to sense, at any given time, thesum (or surface integral) of the intensities of the modulated lightportions within the modulated light stream. (Each time slice of themodulated light stream comprises a spatial ensemble of modulated lightportions due to the modulation unit's action of applying thecorresponding spatial pattern to the light stream.)

In some embodiments, the modulated light stream MLS may be directed ontothe light sensing surface of the light sensing device 130 withoutconcentration, i.e., without decrease in diameter of the modulated lightstream, e.g., by use of photodiode having a large light sensing surface,large enough to contain the cross section of the modulated light streamwithout the modulated light stream being concentrated.

In some embodiments, the optical subsystem 117 may include one or morelenses. FIG. 2E shows an embodiment where optical subsystem 117 isrealized by a lens 117L, e.g., a biconvex lens or a condenser lens.

In some embodiments, the optical subsystem 117 may include one or moremirrors. In one embodiment, the optical subsystem 117 includes aparabolic mirror (or spherical mirror) to concentrate the modulatedlight stream onto a neighborhood (e.g., a small neighborhood) of theparabolic focal point. In this embodiment, the light sensing surface ofthe light sensing device may be positioned at the focal point.

In some embodiments, system 100 may include an optical mechanism (e.g.,an optical mechanism including one or more prisms and/or one or morediffraction gratings) for splitting or separating the modulated lightstream MLS into two or more separate streams (perhaps numerous streams),where each of the streams is confined to a different wavelength range.The separate streams may each be sensed by a separate light sensingdevice. (In some embodiments, the number of wavelength ranges may be,e.g., greater than 8, or greater than 16, or greater than 64, or greaterthan 256, or greater than 1024.) Furthermore, each separate stream maybe directed (e.g., focused or concentrated) onto the corresponding lightsensing device as described above in connection with optical subsystem117. The samples captured from each light sensing device may be used toconstruct a corresponding image for the corresponding wavelength range.In one embodiment, the modulated light stream is separated into red,green and blue streams to support color (R,G,B) measurements. In anotherembodiment, the modulated light stream may be separated into IR, red,green, blue and UV streams to support five-channel multi-spectralimaging: (IR, R, G, B, UV). In some embodiments, the modulated lightstream may be separated into a number of sub-bands (e.g., adjacentsub-bands) within the IR band to support multi-spectral orhyper-spectral IR imaging. In some embodiments, the number of IRsub-bands may be, e.g., greater than 8, or greater than 16, or greaterthan 64, or greater than 256, or greater than 1024. In some embodiments,the modulated light stream may experience two or more stages of spectralseparation. For example, in a first stage the modulated light stream maybe separated into an IR stream confined to the IR band and one or moreadditional streams confined to other bands. In a second stage, the IRstream may be separated into a number of sub-bands (e.g., numeroussub-bands) (e.g., adjacent sub-bands) within the IR band to supportmultispectral or hyper-spectral IR imaging.

In some embodiments, system 100 may include an optical mechanism (e.g.,a mechanism including one or more beam splitters) for splitting orseparating the modulated light stream MLS into two or more separatestreams, e.g., where each of the streams have the same (or approximatelythe same) spectral characteristics or wavelength range. The separatestreams may then pass through respective bandpass filters to obtaincorresponding modified streams, wherein each modified stream isrestricted to a corresponding band of wavelengths. Each of the modifiedstreams may be sensed by a separate light sensing device. (In someembodiments, the number of wavelength bands may be, e.g., greater than8, or greater than 16, or greater than 64, or greater than 256, orgreater than 1024.) Furthermore, each of the modified streams may bedirected (e.g., focused or concentrated) onto the corresponding lightsensing device as described above in connection with optical subsystem117. The samples captured from each light sensing device may be used toconstruct a corresponding image for the corresponding wavelength band.In one embodiment, the modulated light stream is separated into threestreams which are then filtered, respectively, with a red-pass filter, agreen-pass filter and a blue-pass filter. The resulting red, green andblue streams are then respectively detected by three light sensingdevices to support color (R,G,B) acquisition. In another similarembodiment, five streams are generated, filtered with five respectivefilters, and then measured with five respective light sensing devices tosupport (IR, R, G, B, UV) multi-spectral acquisition. In yet anotherembodiment, the modulated light stream of a given band may be separatedinto a number of (e.g., numerous) sub-bands to support multi-spectral orhyper-spectral imaging.

In some embodiments, system 100 may include an optical mechanism forsplitting or separating the modulated light stream MLS into two or moreseparate streams. The separate streams may be directed to (e.g.,concentrated onto) respective light sensing devices. The light sensingdevices may be configured to be sensitive in different wavelengthranges, e.g., by virtue of their different material properties. Samplescaptured from each light sensing device may be used to construct acorresponding image for the corresponding wavelength range.

In some embodiments, system 100 may include a control unit 120configured to supply the spatial patterns to the light modulation unit110, as shown in FIG. 2F. The control unit may itself generate thepatterns or may receive the patterns from some other agent. The controlunit 120 and the light sensing device 130 may be controlled by a commonclock signal so that the light sensing device 130 can coordinate(synchronize) its action of capturing the samples {I_(MLS)(k)} with thecontrol unit's action of supplying spatial patterns to the lightmodulation unit 110. (System 100 may include clock generationcircuitry.)

In some embodiments, the control unit 120 may supply the spatialpatterns to the light modulation unit in a periodic fashion.

The control unit 120 may be a digital circuit or a combination ofdigital circuits. For example, the control unit may include amicroprocessor (or system of interconnected of microprocessors), aprogrammable hardware element such as a field-programmable gate array(FPGA), an application specific integrated circuit (ASIC), or anycombination such elements.

In some embodiments, the control unit 120 may include a random numbergenerator (RNG) or a set of random number generators to generate thespatial patterns or some subset of the spatial patterns.

In some embodiments, system 100 is battery powered. In some embodiments,the system 100 includes a set of one or more solar cells and associatedcircuitry to derive power from sunlight.

In some embodiments, system 100 includes its own light source forilluminating the environment or a target portion of the environment.

In some embodiments, system 100 may include a display (or an interfaceconfigured for coupling to a display) for displaying constructedimages/videos, and/or, for displaying any of the various kinds ofinformation described herein. The display may be based on any of a widevariety of display technologies. In one embodiment, the display may bean LCD display.

In some embodiments, system 100 may include one or more input devices(and/or, one or more interfaces for input devices), e.g., anycombination or subset of the following devices: a set of buttons and/orknobs, a keyboard, a keypad, a mouse, a touch-sensitive pad such as atrackpad, a touch-sensitive display screen, one or more microphones, oneor more temperature sensors, one or more chemical sensors, one or morepressure sensors, one or more accelerometers, one or more orientationsensors (e.g., a three-axis gyroscopic sensor), one or more proximitysensors, one or more antennas, etc.

Regarding the spatial patterns that are used to modulate the lightstream L, it should be understood that there are a wide variety ofpossibilities. In some embodiments, the control unit 120 may beprogrammable so that any desired set of spatial patterns may be used.

In some embodiments, the spatial patterns are binary valued. Such anembodiment may be used, e.g., when the light modulating elements aretwo-state devices. In some embodiments, the spatial patterns are n-statevalued, where each element of each pattern takes one of n states, wheren is an integer greater than two. (Such an embodiment may be used, e.g.,when the light modulating elements are each able to achieve n or moremodulation states). In some embodiments, the spatial patterns are realvalued, e.g., when each of the light modulating elements admits acontinuous range of modulation. (It is noted that even a two-statemodulating element may be made to effectively apply a continuous rangeof modulation by duty cycling the two states during modulationintervals.)

The spatial patterns may belong to a set of measurement vectors that isincoherent with a set of vectors in which the image/video isapproximately sparse (“the sparsity vector set”). (See “Sparse SignalDetection from Incoherent Projections”, Proc. Int. Conf. Acoustics,Speech Signal Processing—ICASSP, May 2006, Duarte et al.) Given two setsof vectors A={a₁} and B={b₁} in the same N-dimensional space, A and Bare said to be incoherent if their coherence measure μ(A,B) issufficiently small. The coherence measure is defined as:

${\mu \left( {A,B} \right)} = {\max\limits_{i,j}{{{\langle{a_{i},b_{j}}\rangle}}.}}$

The number of compressive sensing measurements (i.e., samples of thesequence {I_(MLS)(k)} needed to construct an N-pixel image (or N-voxelvideo sequence) that accurately represents the scene being captured is astrictly increasing function of the coherence between the measurementvector set and the sparsity vector set. Thus, better compression can beachieved with smaller values of the coherence.

In some embodiments, the measurement vector set may be based on a code.Any of various codes from information theory may be used, e.g., codessuch as exponentiated Kerdock codes, exponentiated Delsarte-Goethalscodes, run-length limited codes, LDPC codes, Reed Solomon codes and ReedMuller codes.

In some embodiments, the measurement vector set corresponds to apermuted basis such as a permuted DCT basis or a permuted Walsh-Hadamardbasis, etc.

In some embodiments, the spatial patterns may be random or pseudo-randompatterns, e.g., generated according to a random number generation (RNG)algorithm using one or more seeds. In some embodiments, the elements ofeach pattern are generated by a series of Bernoulli trials, where eachtrial has a probability p of giving the value one and probability 1-p ofgiving the value zero. (For example, in one embodiment p=½.) In someembodiments, the elements of each pattern are generated by a series ofdraws from a Gaussian random variable.)

The system 100 may be configured to operate in a compressive fashion,where the number of the samples {I_(MLS)(k)} captured by the system 100is less than (e.g., much less than) the number of pixels in the image(or video) to be constructed from the samples. In many applications,this compressive realization is very desirable because it saves on powerconsumption, memory utilization and transmission bandwidth consumption.However, non-compressive realizations are contemplated as well.

In some embodiments, the system 100 is configured as a camera or imagerthat captures information representing an image (or a series of images)from the external environment, e.g., an image (or a series of images) ofsome external object or scene. The camera system may take differentforms in different applications domains, e.g., domains such as visiblelight photography, infrared photography, ultraviolet photography,high-speed photography, low-light photography, underwater photography,multi-spectral imaging, hyper-spectral imaging, etc. In someembodiments, system 100 is configured to operate in conjunction with (oras part of) another system, e.g., in conjunction with (or as part of) amicroscope, a telescope, a robot, a security system, a surveillancesystem, a fire sensor, a node in a distributed sensor network, etc.

In some embodiments, system 100 is configured as a spectrometer.

In some embodiments, system 100 is configured as a multi-spectral orhyper-spectral imager.

In some embodiments, system 100 may also be configured to operate as aprojector. Thus, system 100 may include a light source, e.g., a lightsource located at or near a focal point of optical subsystem 117. Inprojection mode, the light modulation unit 110 may be supplied with animage (or a video sequence) so that the image (or video sequence) can bedisplayed on a display surface (e.g., screen).

In some embodiments, system 100 includes an interface for communicatingwith a host computer. The host computer may send control informationand/or program code to the system 100 via the interface. Furthermore,the host computer may receive status information and/or compressivesensing measurements from system 100 via the interface.

In one set of embodiments, a system 160 may be configured as shown inFIG. 2G. System 160 may include the light modulation unit 110 asdescribed above, and may also include an optical subsystem 165, aplurality of light sensing devices (collectively referenced by itemnumber 167), and a sampling subsystem 170. (Furthermore, any subset ofthe features, embodiments and elements described in this patent.)

The light modulation unit 110 is configured to modulate an incidentstream of light L with a sequence of spatial patterns in order toproduce a modulated light stream MLS. The light modulation unit includesan array of light modulating elements. The light modulation unit (LMU)110 may be realized in any of the various ways described herein. Inparticular, the fact that FIG. 2G shows light entering at the left sideof the LMU and exiting the right side is not meant to limit the LMU totransmissive embodiments.

The optical subsystem 165 is configured to deliver light from each of aplurality of spatial subsets of the modulated light stream onto arespective one of the light sensing devices 167. (The linear arrangementof the light sensing devices in FIG. 2G is not meant to be limiting.Various other arrangements, especially two-dimensional arrangements, arepossible.) (The fact that the light sensing devices in FIG. 2G are shownas being separated from each other is not meant to be limiting. In someembodiments, the light sensing devices are physically adjacent (orcontiguous), arranged in an array, or in close proximity to oneanother.) In other words, light from a first subset of the modulatedlight stream is delivered onto a first light sensing device; light froma second subset of the modulated light stream is delivered onto a secondlight sensing device; and so on. The spatial subsets of the modulatedlight stream are produced by respective subsets of the array of lightmodulating elements. Each subset of the array of light modulatingelements produces the respective spatial subset of the modulated lightstream by modulating a respective spatial subset of the incident lightstream. The light sensing devices 165 are configured to generaterespective electrical signals. Each of the electrical signals representsintensity of the respective spatial subset of the modulated light streamas a function of time.

By saying that light from a given spatial subset of the modulated lightstream is “delivered” onto the respective light sensing device meansthat most of the flux (i.e., photons) arriving at the respective lightsensing device comes from the spatial subset. However, it should beunderstood that light crossover may occur in the delivery of light fromspatial subsets of the modulated light stream onto the respective lightsensing devices. Light crossover occurs when a photon of a given spatialsubset arrives a light sensing device other than the respective lightsensing device (e.g., when a photon from the first spatial subsetarrives at the second light sensing device). Light crossover mightoccur, e.g., for photons produced from the light modulation unit alongthe boundaries of the spatial subsets, e.g., due to imperfections in theoptical subsystem 165. Crossover for a given light sensing device mightbe defined as the ratio of the amount of light power received at thelight sensing device from spatial subsets other than the respectivespatial subset to the amount of light power received at the lightsensing device from the respective spatial subset. The crossover for agiven system might be defined, e.g., as the maximum crossover taken overall the light sensing devices.

In different embodiments, different amounts of crossover may betolerated, e.g., depending on the targeted quality of image/videoreconstruction. For example, in different sets of embodiments, themaximum crossover may be required, respectively, to be less than ½, lessthan ¼, less than ⅛, less than 1/32, less than 1/128, less than 1/1024,less than 1/4096, less than 1/16384, less than 2⁻¹⁶.

Furthermore, the “delivery” of light from a given spatial subset of themodulated light stream onto the respective light sensing device does notrequire that 100% of the photons from the spatial subset arrive at therespective light sensing device (or that 100% of the photons that arrivebe converted into electrical charge). Small amounts of light loss may beunavoidable.

The sampling subsystem 170 is configured to acquire samples of theelectrical signals generated respectively by the light sensing devices167. The samples include sample subsets that correspond respectively tothe electrical signals. Each sample subset includes a plurality ofsamples of the respective electrical signal. In the discussion below,the electrical signals are denoted as I₁(t), I₂(t), I_(L)(t), where L isthe number of the light sensing devices, and the sample subsets aredenoted as {I₁(k)}, {I₂(k)}, {I_(L)(k)}. The number L is greater thanone but less than N, where N is the number of light modulating elementsin the array of light modulating elements.

In various embodiments, the number N may take a wide variety of values.For example, in different sets of embodiments, N may be, respectively,in the range [64, 256], in the range [256, 1024], in the range[1024,4096], in the range [2¹²,2¹⁴], in the range [2¹⁴,2¹⁶], in therange [2¹⁶,2¹⁸], in the range [2¹⁸,2²⁰], in the range [2²⁰,2²²], in therange [2²²,2²⁴], in the range [2²⁴,2²⁶], in the range from 2²⁶ toinfinity. The particular value used in any given embodiment may dependon one or more factors specific to the embodiment.

Likewise, the number L may take any of a wide variety of values. Forexample, in different sets of embodiments, L may be, respectively, inthe range [2,4], in the range [4,8], in the range [8,16], in the range[16,32], in the range [32,64], in the range [64, 256], in the range[256,1024], in the range [1024,4096], in the range [2¹²,2¹⁴], in therange [2¹⁴,2¹⁶], in the range [2¹⁶,2¹⁸], in the range [2¹⁸,2²⁰], in therange [2²⁰,2²²], in the range [2²²,2²⁴], in the range [2²⁴,2²⁶].

Furthermore, the ratio N/L make take any of a wide of values, e.g., anyvalue in the range from 2 to N/2. In different sets of embodiments, theratio N/L may be, respectively, in the range [2,4], in the range [4,8],in the range [8,16], in the range [16,64], in the range [64,256], in therange [256,1024], in the range [1024,4096], in the range [4096,2¹⁴], inthe range [2¹⁴,2¹⁶], in the range [2¹⁶,2¹⁸], in the range [2¹⁸,2²⁰], inthe range [2²⁰,2²²], in the range [2²²,2²⁴], in the range from 2²⁴ toinfinity.

As noted above, the samples acquired by the sampling subsystem 170include a sample subset for each of the electrical signals. In someembodiments, each sample subset is acquired over the same window intime, or, approximately the same window in time.

In some embodiments, the samples of each subset are acquired at the samerate. In other embodiments, not all the sample subsets are acquired atthe same rate.

In some embodiments, each of the sample subsets is acquired at the samerate, and, the samples I₁(k), I₂(k), . . . , I_(L)(k) corresponding tosame time index value k are acquired at the same time t_(k), or atleast, close together in time. The samples I₁(k), I₂(k), . . . ,I_(L)(k) corresponding to the same time index value k may be referred toherein as a “sample group”.

In some embodiments, each sample group {I₁(k), I₂(k), I_(L)(k)}corresponds to a respective one of the patterns of the sequence ofspatial patterns. In particular, the sample I_(j)(k), j=1, 2, 3, . . . ,L, may represent an inner product between (a) the restriction of atime-slice of the incident light stream to the j^(th) subset of thearray of light modulating elements and (b) the restriction of a k^(th)one of the spatial patterns to the j^(th) subset.

Each of the L sample subsets is usable to construct a respectivesub-image of an image, as suggested in FIG. 2H. (See also FIG. 2I.) (CAis an acronym for “construction algorithm”.) In other words, the samplesubset {I₁(k)} is usable to construct a first sub-image; the samplesubset {I₂(k)} is usable to construct a second sub-image; and so on.Each sub-image represents a respective one of the spatial subsets of theincident light stream (over the time interval of acquisition). Thesub-images may be joined together to form the image. The image mayrepresent the whole of the incident light stream (or the portion of theincident light stream that impinges upon the array of light modulatingelements) over the time interval of acquisition. The number of pixels ineach sub-image may equal the number of light modulating elements in therespective subset of the array of light modulating elements. Each pixelof the j^(th) sub-image may correspond to a respective one of the lightmodulating elements in the j^(th) subset of the array. Thus, the finalimage may be in an N-pixel image, with each of the N pixelscorresponding to a respective one of the N light modulating elements ofthe array. In some embodiments, each sub-image is N/L pixels in size.However, in other embodiments, the sub-images may be non-identicallysized.

Each sub-image may be constructed using the respective sample subset andthe restriction of the sequence of spatial patterns to the correspondingsubset of the array of light modulating elements. For example, toconstruct the first sub-image, the construction algorithm operates onthe first sample subset (obtained from the first light sensing device)and the spatial patterns restricted to the first subset of the array oflight modulating elements. The construction of the sub-images may beperformed by system 160 and/or by some other system. In the latter case,system 160 may send the sample subsets to the other system. Thesub-images may be constructed using any image construction algorithmknown in the field of compressive sensing, or any combination of suchalgorithms.

In some embodiments, each sample subset is a compressed representationof the corresponding sub-image, i.e., the number of samples m_(SS) inthe sample subset is smaller than the number of pixels n_(PSI) in thecorresponding sub-image. For example, in different embodiments, theratio of m_(SS)/n_(PSI) may be, respectively, less than or equal to 0.8,less than or equal to 0.6, less than or equal to 0.4, less than or equalto 0.3, less than or equal to 0.2, less than or equal to 0.1, less thanor equal to 0.05, less than or equal to 0.025. In some embodiments, nocompression is achieved, i.e., m_(SS) is greater than or equal ton_(PSI). (Such might be the case when the ratio N/L is small or when thesignal-to-noise ratio is small.)

FIG. 2I illustrates one embodiment of the processes of sampleacquisition and image construction in a case (L=4) where there are fourlight sensing devices LSD1-LSD4. At the left is a visualization of themodulated light stream at the modulating plane of the light modulationunit 110 (at a snapshot in time). The light modulation unit includes Nlight modulating elements. The four light sensing devices receive lightfrom four respective quadrants of the modulation plane. The electricalsignal generated by each light sensing device is sampled to generate arespective sample subset. (See the A/D convert blocks.) Each samplesubset is processed by a construction algorithm (CA) to generate arespective sub-image. (The construction algorithm may be anyconstruction algorithm known in the field of compressive sensing.) Thesub-images are stitched together to form an N-pixel image.

For more information on various embodiments of system 160, please seeU.S. patent application Ser. No. 13/197,304, filed on Aug. 3, 2011,entitled “Decreasing Image Acquisition Time for Compressive ImagingDevices”, invented by Woods et al., which is hereby incorporated byreference in its entirety as though fully and completely includedherein. (In the parlance of that patent application, system 160 isreferred to as “system 600”.)

In one realization 200 of system 100, the light modulation unit 110 maybe realized by a plurality of mirrors, e.g., as shown in FIG. 3A. (Themirrors are collectively indicated by the label 110M.) The mirrors 110Mare configured to receive corresponding portions of the light L receivedfrom the environment, albeit not necessarily directly from theenvironment. (There may be one or more optical elements, e.g., one ormore lenses along the input path to the mirrors 110M.) Each of themirrors is configured to controllably switch between two orientationstates. In addition, each of the mirrors is configured to (a) reflectthe corresponding portion of the light onto a sensing path 115 when themirror is in a first of the two orientation states and (b) reflect thecorresponding portion of the light away from the sensing path when themirror is in a second of the two orientation states.

In some embodiments, the mirrors 110M are arranged in an array, e.g., atwo-dimensional array or a one-dimensional array. Any of various arraygeometries are contemplated. For example, in different embodiments, thearray may be a square array, a rectangular array, a hexagonal array,etc. In some embodiments, the mirrors are arranged in a spatially-randomfashion.

The mirrors 110M may be part of a digital micromirror device (DMD). Forexample, in some embodiments, one of the DMDs manufactured by TexasInstruments may be used.

The control unit 120 may be configured to drive the orientation statesof the mirrors through the series of spatial patterns, where each of thepatterns of the series specifies an orientation state for each of themirrors.

The light sensing device 130 may be configured to receive the lightportions reflected at any given time onto the sensing path 115 by thesubset of mirrors in the first orientation state and to generate ananalog electrical signal representing I_(MLS)(t) representing acumulative intensity of the received light portions as function of time.As the mirrors are driven through the series of spatial patterns, thesubset of mirrors in the first orientation state will vary from onespatial pattern to the next. Thus, the cumulative intensity of lightportions reflected onto the sensing path 115 and arriving at the lightsensing device will vary as a function time. Note that the term“cumulative” is meant to suggest a summation (spatial integration) overthe light portions arriving at the light sensing device at any giventime. This summation may be implemented, at least in part, optically(e.g., by means of a lens and/or mirror that concentrates or focuses thelight portions onto a concentrated area as described above).

System realization 200 may include any subset of the features,embodiments and elements discussed above with respect to system 100. Forexample, system realization 200 may include the optical subsystem 105 tooperate on the incoming light L before it arrives at the mirrors 110M,e.g., as shown in FIG. 3B.

In some embodiments, system realization 200 may include the opticalsubsystem 117 along the sensing path as shown in FIG. 4. The opticalsubsystem 117 receives the light portions reflected onto the sensingpath 115 and directs (e.g., focuses or concentrates) the received lightportions onto a light sensing surface (or surfaces) of the light sensingdevice 130. In one embodiment, the optical subsystem 117 may include alens 117L, e.g., as shown in FIG. 5A.

In some embodiments, the optical subsystem 117 may include one or moremirrors, e.g., a mirror 117M as shown in FIG. 5B. Thus, the sensing pathmay be a bent path having more than one segment. FIG. 5B also shows onepossible embodiment of optical subsystem 105, as a lens 105L.

In some embodiments, there may be one or more optical elementsintervening between the optical subsystem 105 and the mirrors 110M. Forexample, as shown in FIG. 5C, a TIR prism pair 107 may be positionedbetween the optical subsystem 105 and the mirrors 110M. (TIR is anacronym for “total internal reflection”.) Light from optical subsystem105 is transmitted through the TIR prism pair and then interacts withthe mirrors 110M. After having interacted with the mirrors 110M, lightportions from mirrors in the first orientation state are reflected by asecond prism of the pair onto the sensing path 115. Light portions frommirrors in the second orientation state may be reflected away from thesensing path.

Scanning a Region Along a 1D Path

In one set of embodiments, a method 600 for focusing a compressiveimaging device may include the actions shown in FIG. 6. A compressiveimaging (CI) device is a device that captures a compressedrepresentation(s) of an image(s) using a light modulation unit and alight sensing device, as variously described above. (FIGS. 2A through 5Cillustrate different embodiments of a CI device.) (The set of samplescaptured from the light sensing device is the compressed representationof the image.) In some embodiments, the CI device also includes theoptical subsystem 105 as described above. The optical subsystem 105 mayhave an adjustable focus setting, e.g., as is typical with many cameralens units.

Action 610 includes supplying a sequence of calibration patterns to alight modulation unit of the CI device, e.g., the light modulation unit110 as variously described above. The light modulation unit includes anarray of light modulating elements, e.g., as variously described above.The calibration patterns are intended to facilitate the process offocusing the CI device.

Action 615 includes modulating an incident light stream L with thesequence of calibration patterns to produce a modulated light streamMLS. (The light modulation unit performs this modulation operation.)(See, e.g., FIGS. 2A through 5C.) The sequence of calibrations patternsis configured to effect a movement of a region along a one-dimensionalpath on the array of light modulating elements. Each of the calibrationpatterns corresponds a different position of the region along theone-dimensional path.

The region may be a small region, e.g., a region comprising less than orequal to 128 light modulating elements in diameter. For example, indifferent embodiments, the region may be, respectively, a 1×1 region, a1×2 region, a 2×1 region, a 2×2 region, a 2×3 region, a 3×2 region, a2×4 region, a 4×2 region, a 4×4 region, etc. (The dimensions are interms of number of light modulating elements covered.) More generally,the region may be a k_(X)×k_(Y) region with k=k_(X)×k_(Y) being apositive integer. In different sets of embodiments, the integer k maybe, respectively, less than or equal to 512, less than or equal to 256,less than or equal to 128, less than or equal to 64, less than or equalto 32, less than or equal to 16, less than or equal to 8, less than orequal to 4.

In the one-dimensional path may take any of a wide variety of forms. Forexample, in different embodiments, the path may be line segment, apolyline, a circle, a triangle, a square, an arbitrary curve, a diamond,a hexagon, etc.

At 620, a light sensing device of the CI device (e.g., light sensingdevice 130 as variously described above) receives the modulated lightstream MLS and acquires a sequence of samples {I_(MLS)(k)} representingan intensity of the modulated light stream as a function of time. (Thelight sensing device may include the A/D converter 140 as describedabove.) The sequence of samples includes at least one sample for each ofthe calibration patterns. For example, the operation of the control unit120 and A/D converter 140 may be coordinated to ensure this condition.

Action 625 includes computing a focus indicator value based on thesequence of samples. This computing action may be performed byprocessing unit 150 as described above. The focus indicator valueindicates an extent to which the incident light stream is in focus alongthe one-dimensional path on the array of light modulating elements. Thesequence of samples may be interpreted as (or may be condensed so as torepresent) a one-dimensional image with position along the imagecorresponding to position along the one-dimensional path on the array oflight modulating elements.

The focus indicator value may be computed by evaluating a conventional1D focus metric (of which there are many known in the art) on thesequence of samples. In some embodiments, two or more conventional 1Dfocus metrics may be evaluated and combined (for example, by averaging,or taking the maximum) to obtain a composite focus metric.

In some embodiments, the focus metric may involve: a measure offrequency content (such as might be derived from a Fourier transform ofthe image), global variance, Sobel Tenenbaum's Algorithm, Boddeke'sAlgorithm, any estimate of the second derivative (e.g., using theLaplacian Operator or a numerical second difference), an estimate of thegradient in the image, an estimate of the variance or a function on thedata such as the variance of the gradient or variance of the secondderivative, a threshold absolute gradient and/or threshold squaredgradient, Tenenbaum gradient, etc. (This list is not meant to belimiting. Indeed, a wide variety of other focus metrics may be used.)

In some embodiments, the CI device includes the optical subsystem 105.(See, e.g., FIGS. 2C-2E and FIGS. 3B through 5C.) Optical subsystem 105receives and operates on the incident light stream prior to the incidentlight stream arriving at the light modulation unit. It is desirable foroptical subsystem 105 to operate on the incident light stream in such asway that the incident light stream comes into focus at the array oflight modulating elements. However, that condition may fail to holdwhen, e.g., the external object (that is being imaged) moves, or whenthe CI device is moved with respect to the external object. The presentmethod may be used to restore that condition of focus.

In some embodiments, the method 600 may also include changing a focussetting of the optical subsystem 105 based on data including the focusindicator value. The data may also include a current focus setting, aprevious focus setting, and a previous focus indicator valuecorresponding to the previous focus setting. Processing unit 150 mayoperate on the data to determine an amount (direction and magnitude) tochange the focus setting (e.g., using the Newton-Raphson method).

In some embodiments, the method 600 may include repeatedly performing aset of operations. The set of operations may include: changing the focussetting of the optical subsystem 105; and performing actions 610 through625. The set of operations may be repeated until the focus indicatorvalue becomes less than a threshold value. The threshold value isselected so that the final focus setting is close enough to the optimalsetting. “Close enough” might have different meanings in differentapplication contexts, or in different modes of operation. The thresholdvalue may be a programmable parameter.

In some embodiments, the method 600 may also include changing an opticaldistance between the light modulation unit 110 and the optical subsystem105 based on data including the focus indicator value. The data may alsoinclude a current value of the optical distance, a previous value of theoptical distance, and a previous focus indicator value corresponding tothe previous optical distance value. Changing the distance between thelight modulation unit and the optical subsystem may involve translatingthe light modulation unit, e.g., along the optical path of the incidentlight stream.

The term “optical distance” is used as a synonym for “optical pathlength” and refers to the length of travel of a light stream (or beam)between two given elements of a system. Optical distance is notnecessarily the same as the straight-line distance between the twoelements since the light stream travelling between the two elements neednot follow a straight-line path. For example, optical devices such asmirrors, beam splitters, lenses, prisms, etc. may be used direct theflow of a light stream along any desired path.

In some embodiments, the light modulation unit and the light sensingdevice (and any optics intervening between the light modulation unit andthe light sensing device) may be configured to translate together, as aunit. For example, those elements may be mounted on a translatableplatform. The platform may be translated relative to the opticalsubsystem 105. In one such embodiment, the light sensing device may beconfigured to additionally translate with respect to the platform.

In some embodiments, the light modulation unit may be configured totranslate independently of other elements of the system (the CI device).

In some embodiments, the method 600 may also include repeatedlyperforming a set of operations, where the set of operations includes:changing an optical distance d_(OM) between the light modulation unitand the optical subsystem; and performing actions 610 through 625. Theset of operations may be repeated until the focus indicator valuebecomes less than a threshold value. The comments above regarding thethreshold value may apply here as well.

In some embodiments, the method 600 may also include changing an opticaldistance d_(MS) (optical path length) between the light modulation unitand the light sensing device based on a final optical distanced_(OM(final)) between the light modulation unit and the opticalsubsystem after the focus indicator value becomes less than thethreshold value. The functional relationship between d_(OM) and d_(MS)may be known by calibration.

Adaptively Adding Another 1D Path

In some embodiments, method 600 may also include: determining that thefocus indicator value computed at 625 is inaccurate (by any of variousmeans), and then performing actions 610 through 625 a second time, usinga second sequence of calibration patterns, to obtain a second sequenceof samples and a second focus indicator value. The second sequence ofcalibration patterns is configured to effect the movement of the regionalong a second one-dimensional path on the array of light modulatingelements (i.e., different from the one-dimensional path used in thefirst execution of action 615). Hopefully, the second one-dimensionalpath will cut through a portion of the incident light stream that hasmore dynamic information content, and thus, give rise to a more accuratefocus indicator value in the second iteration. If that second focusindicator value is found to be inaccurate, a third iteration of 610through 625 may be performed, and so on.

The original focus indicator value may be determined to be inaccurate bycomputing a measure of the dynamic variation of the original sequence ofsamples and determining that the measure is smaller than a threshold.For example, if the RMS value (or the average of the absolute value) ofthe second derivative of the sample sequence is smaller than thethreshold, the focus indicator value may be declared to be inaccurate.As another example, if the standard deviation of the sample sequence istoo small, the focus indicator value may be declared to be inaccurate.There are a wide variety of ways known in the field of image processingfor measuring the dynamic variation of a sequence (or 1D image), and anyof those methods may be used here.

A Plurality of 1D Paths for the Current State of Focus

In some embodiments, the method 600 may also include performing actions610 through 625 a plurality of times, using a respective plurality ofcalibration sequences to obtain a respective plurality of focusindicator values for a respective plurality of one-dimensional paths onthe array of light modulating elements. (Processing unit 150 may directthe plurality of iterations.) The one-dimensional paths are differentpaths, i.e., the set of light modulating elements covered by any one ofthem is not the same as the set of light modulating elements covered byany other one of them. In other words, no two of the paths are entirelycoincident (sitting on top of each other). Note however that theone-dimensional paths are allowed to intersect in some embodiments. (Theoptical distance d_(OM) and the focus setting of the optical subsystem105 may be held constant during the plurality of iterations of 610through 625 so that the resulting focus indicator values are indicativeof the current state of focus of the incident light stream.) Using aplurality of one-dimensional paths increases the probability that at oneof them will cut through a spatially dynamic portion (e.g., one or moreedges) in the image carried by the incident light field. The processingunit 150 may operate on the plurality of focus indicator values todetermine a representative focus indicator value for a current state offocus of the incident light stream on the array of light modulatingelements. For example, the focus indicator value corresponding to thepath (1-D image) with the highest measure of dynamic variability may beselected as the representative.

Adaptive Algorithm for Locating Improved (or Optimal) Focus

In some embodiments, the method 600 may also include: directing anactuator to change an optical distance between the optical subsystem 105and the light modulation unit 110 according to a first displacementvalue; performing actions 610 through 625 a second time, to obtain anadditional focus indicator value for an additional sequence of samples(e.g., using the same one-dimensional path as in the first iteration ofactions 610 through 625); and computing a second displacement value forthe optical distance based on data including the focus indicator valuefor the original sequence of samples, the focus indicator value for theadditional sequence of samples and the first displacement value. In someembodiments, the data may also include past-history information, e.g.,focus indicator values and corresponding displacement values from one ormore previous states of the optical distance. (In some embodiments, theactuator allows variable-sized steps. In other embodiments, the actuatormay allow only unit-sized steps.) The method 600 may also includedirecting the actuator to change the optical distance according to thesecond displacement value.

In some embodiments, the method 600 may also include: directing anactuator to change a focus setting of the optical subsystem 105according to a first displacement value; performing actions 610 through625 a second time, to obtain an additional focus indicator value for anadditional sequence of samples (e.g., for the same one-dimensionalpath); and computing a second displacement value for the focus settingbased on data including the focus indicator value for the originalsequence of samples, the focus indicator value for the additionalsequence of samples and the first displacement value. In someembodiments, the data may also include past-history information, e.g.,focus indicator values and corresponding displacement values from one ormore previous states of the focus setting. The method 600 may alsoinclude directing the actuator to change the focus setting of theoptical subsystem 105 according to the second displacement value.

Batch Algorithm

In some embodiments, method 600 may also include: directing an actuatorto change an optical distance between the optical subsystem 105 and thelight modulation unit 110 through a range; performing actions 610through 625 for each of a plurality of optical distances within therange in order to obtain a corresponding plurality of sample sequencesand a corresponding plurality of focus indicator values (e.g., using thesame one-dimensional path for each of the plurality of distances);determining an optimal value for the optical distance based on ananalysis of data including the plurality of focus indicator values. Theoptimal value may be determined maximizing (or minimizing) the functionof optical distance defined by the focus indicator values. Numericalmethods for finding the minimum or maximum of a function are well known.Any such method may be used here. The actuator may then be directed tochange the optical distance to the optimal value.

In some embodiments, the method 600 may also include: directing anactuator to change a focus setting of the optical subsystem 105 througha range of settings; performing actions 610 through 625 for each of aplurality of settings within the range in order to obtain acorresponding plurality of sample sequences and a correspondingplurality of focus indicator values (e.g., using the sameone-dimensional path for each of the plurality of settings); anddetermining an optimal value for the focus setting based on an analysisof data including the plurality of focus indicator values, e.g., bycomputing the minimum or maximum of the function of focus settingdefined by the focus indicator values. The method 600 may also includedirecting the actuator to change the focus setting to the optimal value.

In some embodiments, the light sensing device 130 includes a pluralityof light sensing elements. (In some embodiments, the light sensingelements are organized in an array.) Each of the light sensing elementsgenerates a respective electrical signal representing intensity of arespective spatial portion of the modulated light stream. The action ofacquiring the sequence of samples (referred to at 620 of FIG. 6) mayinvolve adding the electrical signals to obtain a sum signal andsampling the sum signal. For more information on how to deliver spatialportions of the modulated light stream onto respective light sensingelements, please see U.S. patent application Ser. No. 13/197,304, filedon Aug. 3, 2011, entitled “Decreasing Image Acquisition Time forCompressive Imaging Devices”, invented by Woods et al., which is herebyincorporated by reference in its entirety as though fully and completelyincluded herein.

In some embodiments, instead of summing the electrical signals (from therespective light sensing elements) in the analog domain, they are summedin the digital domain. Thus, the action of acquiring the sequence ofsamples (referred to at 620 of FIG. 6) may include sampling each of theelectrical signals and adding the sampled versions of the electricalsignals in the digital domain.

In some embodiments, the light sensing device 130 includes only onelight sensing element (e.g., an elementary transducer between lightenergy and electrical energy).

In some embodiments, the action of computing a focus indicator valueincludes: computing a discrete Fourier transform of the sequence ofsamples {I_(MLS)(k)} (or, a one-dimensional image derived from sequenceof samples); and computing an amount of energy present in the discreteFourier transform at frequencies above a cutoff frequency. In someembodiments, the computed energy may be one of a plurality of factorsthat determine the focus indicator value.

Method 700

In one set of embodiments, a method 700 for focusing a compressiveimaging (CI) device may include the actions shown in FIG. 7.(Furthermore, method 700 may also include any subset of the actions,embodiments and features described above in connection with method 600.)

Action 710 includes supplying a sequence of calibration patterns to alight modulation unit of the CI device (e.g., the light modulation unit110), where the light modulation unit includes an array of lightmodulating elements, e.g., as variously described above.

Action 715 includes modulating an incident light stream with thesequence of calibration patterns to produce a modulated light stream,e.g., as variously described above. (The modulating is performed by thelight modulation unit.) The sequence of calibrations patterns isconfigured to effect a movement of a region along a one-dimensional pathon the array of light modulating elements, where each of the calibrationpatterns corresponds a different position of the region along theone-dimensional path, e.g., as variously described above.

Action 720 includes acquiring a sequence of the frames from an array oflight sensing elements, wherein each of the light sensing elements isconfigured to receive a respective spatial portion of the modulatedlight stream, wherein each of the frames of the acquired sequencecorresponds to a respective one of the calibration patterns, whereineach of the frames includes one sample from each of the light sensingelements. In some embodiments, the CI device may include a readoutintegrated circuit to read frames of samples from the light sensingarray. See U.S. application Ser. No. 13/197,304 for more information oncircuitry for reading frames of samples (or “groups of samples in theparlance of that application) from a light sensing array.

Action 725 includes determining a sequence of sum values, wherein eachof the sum values is determined by adding the samples in a respectiveone of the frames of the acquired sequence. The sum is over the array oflight sensing elements. Processing unit 150 may be configured toperforming this summing action.

Action 730 includes operating on the sequence of sum values to determinea focus indicator value, wherein the focus indicator value indicates anextent to which the incident light stream is in focus along theone-dimensional path.

Method 800

In one set of embodiments, a method 800 for focusing a compressiveimaging (CI) device may include the actions shown in FIG. 8.(Furthermore, method 800 may also include any subset of the actions,embodiments and features described above in connection with method 600and method 700.)

Action 810 includes supplying a sequence of calibration patterns to alight modulation unit of the CI device, e.g., as variously describedabove. The light modulation unit includes an array of light modulatingelements, e.g., as variously described above.

Action 815 includes modulating an incident light stream with thesequence of calibration patterns to produce a modulated light stream,e.g., as variously described above.

At 820, a light sensing device of the CI device (e.g., light sensingdevice 130) receives the modulated light stream and acquires a sequenceof samples representing an intensity of the modulated light stream as afunction of time, where the sequence of samples includes at least onesample for each of the calibration patterns.

Action 825 includes changing a focus setting of the CI device based onthe sequence of samples.

In some embodiments, the focus setting of the CI device includes anoptical distance between an optical subsystem of the CI device (e.g.,the optical subsystem 105 described above) and the light modulation unit110.

In some embodiments, the focus setting of the CI device includes a focussetting of an optical subsystem of the CI device (e.g., of the opticalsubsystem 105 described above).

In some embodiments, the method 800 may include repeating actions 810through 825 until a focus metric based on the sequence of samples (inthe most recent iteration of 810 through 825) is optimized.

In some embodiments, the sequence of calibrations patterns is configuredto effect the movement of a region along a one-dimensional path on asurface of the light modulation unit, wherein each of the calibrationpatterns corresponds a different position of the region along theone-dimensional path.

Display Acquired 1-D Image and Let User Control Focus Adjustment

In one set of embodiments, a method 900 for focusing a compressiveimaging (CI) device may include the actions shown in FIG. 9.(Furthermore, method 900 may also include any subset of the actions,embodiments and features described above in connection with method 600,method 700 and method 800.)

Action 910 includes supplying a sequence of calibration patterns to alight modulation unit of the CI device, wherein the light modulationunit includes an array of light modulating elements, e.g., as variouslydescribed above.

Action 915 includes modulating an incident light stream with thesequence of calibration patterns to produce a modulated light stream,e.g., as variously described above. The sequence of calibrationspatterns is configured to effect a movement of a region along aone-dimensional path on the light modulation unit, wherein each of thecalibration patterns corresponds a different position of the regionalong the one-dimensional path, e.g., as variously described above.

At 920, a light sensing device of the CI device receives the modulatedlight stream and acquires a sequence of samples representing anintensity of the modulated light stream as a function of time, where thesequence of samples includes at least one sample for each of thecalibration patterns.

Action 925 includes displaying a one-dimensional image on a display(e.g., a display as variously described above). The one-dimensionalimage is based on the sequence of samples. In cases where one exactlyone sample is acquired per calibration pattern, the sequence of samplesmay serve as the one-dimensional image. In cases where two or moresamples are acquired per calibration pattern, the two or more samplesper calibration pattern may be reduced to one refined sample percalibration pattern, e.g., by averaging the two or more samples, and/or,discarding any of the two or more samples that correspond topattern-transition transients. The refined samples, one per calibrationpattern, may be used as the one-dimensional image. See U.S. applicationSer. No. 13/207,258 for more information on such averaging.

Action 930 includes adjusting a focus setting of the CI device based onuser input. The user input may specify the direction and/or magnitude ofthe focus setting adjustment. For example, the CI device (e.g., system100 in any of its various embodiments) may include a graphical userinterface (GUI) or other interface to allow the user to specify focusadjustments. In some embodiments, the CI device may include a set of oneor more physical input devices such as buttons, knobs, sliders anddials, or, one or more GUI equivalents thereof.

In some embodiments, the focus setting of the CI device is a focussetting of an optical subsystem of the CI device (e.g., opticalsubsystem 105).

In some embodiments, the focus setting of the CI device is an opticaldistance between an optical subsystem of the CI device (e.g., opticalsubsystem 105) and the light modulation unit.

In some embodiments, method 900 may include performing actions 910through 930 a plurality of times.

In some embodiments, method 900 may include receiving user inputspecifying a position and/or orientation of the one-dimensional path onthe array of light modulating elements.

Focusing External Light on the Modulator Using Reconstructed Images

In one set of embodiments, a method 1000 focusing a compressive imaging(CI) device may include the actions shown in FIG. 10. (Furthermore,method 1000 may include any subset of the actions, embodiments andfeatures described above in connection with methods 600, 700, 800 and900.)

Action 1010 includes supplying a sequence of spatial patterns to a lightmodulation unit of the CI device (e.g., the light modulation unit 110 asdescribed above). The light modulation unit includes an array of lightmodulating elements, e.g., as variously described above.

Action 1015 includes modulating an incident light stream with thesequence of spatial patterns to produce a modulated light stream, e.g.,as variously described above. (The modulating action is performed by thelight modulation unit.)

At 1020, a light sensing device of the CI device (e.g., the lightsensing device 130 as described above) receives the modulated lightstream and acquires a sequence of samples {I(k)} representing anintensity of the modulated light stream as a function of time, e.g., asvariously described above. The sequence of samples includes at least onesample for each of the spatial patterns.

Action 1025 includes constructing an image using the sequence of spatialpatterns and the acquired samples, e.g., using any image constructionalgorithm known in the field compressive sensing. The spatial patternsused to modulate the incident light stream are preferably drawn from aset of measurement vectors that is incoherent with respect to a sparsityvector set that is used to perform the image construction. The image maybe two-dimensional image. (However, in alternative embodiments, theimage may be a one-dimensional image.)

In some embodiments, the image is an n-pixel image with n less than thenumber N of light modulating elements in the light modulation unit, andthe number m of spatial patterns in the sequence of spatial patterns isless than n. The compression ratio m/n may be different in differentembodiments. For example, in different sets of embodiments, thecompression ratio m/n may, respectively be less than or equal to 0.9,less than or equal to 0.8, less than or equal to 0.7, less than or equalto 0.6, less than or equal to 0.5, less than or equal to 0.4, less thanor equal to 0.3, less than or equal to 0.2, less than or equal to 0.1,less than or equal to 0.05, less than or equal to 0.025.

Action 1030 includes computing a focus indicator value based on theconstructed image, where the focus indicator value indicates an extentto which the incident light stream is in focus at the light modulationunit (i.e., the array of light modulating elements).

In some embodiments, the focus indicator may be computed by evaluating afocus metric on the image, e.g., a focus metric based on a measure offrequency content (such as might be derived from a Fourier transform ofthe image), global variance, Sobel Tenenbaum's Algorithm, Boddeke'sAlgorithm, any estimate of the second derivative (e.g., using theLaplacian Operator or a numerical second difference), an estimate of thegradient in the image, an estimate of the variance or a function on thedata such as the variance of the gradient or variance of the secondderivative, a threshold absolute gradient and/or threshold squaredgradient, Tenenbaum gradient, etc. (This list is not meant to belimiting. Indeed, a wide variety of other focus metrics may be used.) Insome embodiments, the focus indicator value may be a combination (e.g.,a functional combination or average or minimum or maximum or astatistic) of a combination of two or more focus metrics evaluated onthe image.

In some embodiments, each of the spatial patterns is restricted to asubset S of the array of light modulating elements. By restricting thespatial patterns to a subset of the light modulating array, the numberof spatial patterns m_(R) (and corresponding samples) required toconstruct the image (of action 1025) decreases with respect to thenumber that would be required if the spatial patterns were free to varyover the whole light modulating array. Here the term “restriction” of aspatial patterns to a subset S of the light modulating array means thatthe spatial pattern is null (or zero) outside the subset S.

In some embodiments, the subset S is a contiguous two-dimensional regionof the array. (However, in alternative embodiments, the subset S may bea one-dimensional region.)

In some embodiments, the subset S is a union of contiguous regions ofthe light modulating array.

In some embodiments, the subset S is a user-specified region ofinterest. The CI device may include a graphical user interface forspecifying the region of interest.

In some embodiments, the subset S corresponds to a region of interestdetermined based on an analysis of a previously-acquired image (i.e., animage that has been previously constructed based on previously acquiredsamples.)

In some embodiments, the subset S is a convex region located at thecenter of the light modulating array.

Method 1000 may be performed to enable focusing of the CI device. Thecompression ratio m/n used when executing method 1000 for focusing basedon subset S may the same as (or on the same order of magnitude) as thecompression ratio M/N used when acquiring full scale images, e.g.,during normal operational mode as a compressive imager. Larger values ofM/N imply reconstructed images of higher quality, and thus, smalldeviations from best focus may be more perceptible. Thus, when the CIdevice uses larger values of M/N, it may also use larger values of m/nduring the focusing procedure based on subset S, so that the focus imagegenerated at 1025 may be of commensurate quality. Thus, any focusingprocedure based on the focus image will be able to converge closer tothe state of best focus.

In some embodiments, the CI device includes the optical subsystem 105 asdescribed above. The optical subsystem 105 receives and operates on theincident light stream prior to the incident light stream arriving at thelight modulation unit.

In some embodiments, method 1000 may include changing a focus setting ofthe optical subsystem 105 based on data including the focus indicatorvalue.

In some embodiments, method 1000 may include repeatedly performing a setof operations, where the set of operations includes: changing a focussetting of the optical subsystem 105; and performing actions 1010through 1030. The set of operations may be repeated until the focusindicator value becomes less than a threshold value. The threshold valuemay be different in different embodiments.

In some embodiments, the method 1000 may include changing an opticaldistance between the optical subsystem 105 and the light modulation unit110 (i.e., the optical path length of the incident light stream betweenthe optical subsystem 105 and the light modulation unit 110) based ondata including the focus indicator value. As described above, theoptical distance may be changed by translating the light modulation unit110. Alternatively, the optical distance may be changed by translatingthe optical subsystem 105.

In some embodiments, the method 1000 may include repeatedly performing aset of operations, where the set of operations includes: changing anoptical distance between the optical subsystem 105 and the lightmodulation unit 110; and performing actions 1010 through 1030. The setof operations may be repeated until the focus indicator value becomesless than a threshold value.

In some embodiments, the method 1000 may include changing an opticalpath length between the light modulation unit and the light sensingdevice based on a final distance between the light modulation unit andthe optical subsystem after the focus indicator value becomes less thanthe threshold value.

In some embodiments, the method 1000 may include changing an angle oforientation of the light modulation unit based on data including thefocus indicator value.

In some embodiments, the method 1000 may also include: directing anactuator to change an optical distance between the optical subsystem andthe light modulation unit according to a first displacement value;performing actions 1010 through 1030 a second time (e.g., using the samesubset S of the light modulating array), in order to obtain anadditional image and a focus indicator value for the additional image;and computing a second displacement value for the optical distance basedon data including the focus indicator value for the image, the focusindicator value for the additional image and the first displacementvalue. (As described above, the displacement values may variable in sizeand direction, or, perhaps only in direction.) The method 1000 may alsoinclude directing the actuator to change the optical distance accordingto the second displacement value.

In some embodiments, the method 1000 may also include: directing anactuator to change a focus setting of the optical subsystem 105according to a first displacement value; performing actions 1010 through1030 a second time (e.g., using the same subset S of the lightmodulating array), in order to obtain an additional image and a focusindicator value for the additional image; and computing a seconddisplacement value for the focus setting based on data including thefocus indicator value for the image, the focus indicator value for theadditional image and the first displacement value. The method 1000 mayalso include directing the actuator to change the focus setting of theoptical subsystem according to the second displacement value.

In some embodiments, the method 1000 may also include: directing anactuator to change an optical distance between the optical subsystem andthe light modulation unit through a range of distances; performingactions 1010 through 1030 for each of a plurality of optical distanceswithin the range in order to obtain a corresponding plurality of imagesand a corresponding plurality of focus indicator values (e.g., with thearray subset S being held constant for the plurality of distances); anddetermining an optimal value for the optical distance based on ananalysis of data including the focus indicator values (and in someembodiments, also the corresponding optical distances). As describedabove, the optimal value may be determined by maximizing (or minimizing)the function of optical distance defined by the focus indicator values.The actuator may then be directed to change the optical distance to theoptimal value.

In some embodiments, the method 1000 may also include: directing anactuator to change the focus setting of the optical subsystem 105through a range of settings; performing actions 1010 through 1030 foreach of a plurality of focus settings within the range in order toobtain a corresponding plurality of images and a corresponding pluralityof focus indicator values (e.g., with the array subset S being heldconstant for the plurality of settings); and determining an optimalvalue for the focus setting based on an analysis of data including theplurality of focus indicator values (and in some embodiments, also thecorresponding focus settings). As described above, the optimal value maybe determined by maximizing (or minimizing) the function of focussetting defined by the focus indicator values. The actuator may then bedirected to change the focus setting to the optimal value.

In some embodiments, the light sensing device includes a plurality oflight sensing elements, where each of the light sensing elementsgenerates a respective electrical signal representing intensity of arespective spatial portion of the modulated light stream. The action ofacquiring the sequence of samples {I(k)} (referred to at 1020 of FIG.10) may include adding the electrical signals to obtain a sum signal andsampling the sum signal. In one embodiment, the CI device includes oneA/D converter to sample the sum signal and also one A/D converter perlight sensing element (to sample the respective electrical signals).Thus, with L light sensing elements, the CI device may include L+1 A/Dconverters. However, other embodiments may include fewer A/D converters.

In some embodiments, the action of acquiring the sequence of samplescomprises sampling each of the electrical signals and adding the sampledversions of the electrical signals in the digital domain.

In some embodiments, the light sensing device includes only one lightsensing element.

Method 1100

In one set of embodiments, a method 1100 may include the actions shownin FIG. 11. (Furthermore, method 1100 may include any subset of theactions, embodiments and features described above in connection withmethods 600, 700, 800, 900 and 1000.)

Action 1110 may include supplying a sequence of spatial patterns to alight modulation unit of the CI device (e.g., the light modulation unit110 as described above).

Action 1115 may include modulating an incident light stream with thesequence of spatial patterns to produce a modulated light stream. (Themodulating action is performed by the light modulation unit.)

Action 1120 may include acquiring a sequence of the frames from an arrayof light sensing elements of the CI device. Each of the light sensingelements receives a respective spatial portion of the modulated lightstream. Each of the frames corresponds to respective one of the spatialpatterns. Furthermore, each of the frames includes one sample from eachof the light sensing elements.

Action 1125 may include determining a sequence of sum values, whereineach of the sum values is determined by adding the samples in arespective one of the frames of the acquired sequence.

Action 1130 may include constructing an image using the sequence ofspatial patterns and the sequence of sum values, e.g., using any imagereconstruction algorithm known in the field of compressive sensing.

Action 1135 may include computing a focus indicator value based on theimage, where the focus indicator value indicates an extent to which theincident light stream is in focus at the light modulation unit (or infocus within the portion of the light modulation unit corresponding tothe image).

Method 1200

In one set of embodiments, a method 1200 for determining focus-relatedinformation for a compressive imaging (CI) device includes the actionsshown in FIG. 12. (Furthermore, method 1200 may include any subset ofthe actions, embodiments and features described above in connection withmethods 600 through 1100.)

Action 1210 includes modulating an incident light stream with a sequenceof spatial patterns, thereby generating a modulated light stream, e.g.,as variously described above. The modulation is performed by a lightmodulation unit of the CI device (e.g., by the light modulation unit 110as described above).

Action 1215 includes generating an electrical signal representingintensity of the modulated light stream as a function of time, where theelectrical signal is generated by a light sensing device of the CIdevice (e.g., by the light sensing device 130 as variously describedabove).

Action 1220 includes acquiring samples of the electrical signal, e.g.,as variously described above.

Action 1225 includes constructing an image using the sequence of spatialpatterns and a subset of the acquired samples that corresponds to thesequence of spatial patterns. The subset includes at least one samplefor each of the spatial patterns.

Action 1230 includes computing focus information based on the image,wherein the focus information comprises information regarding at extentto which the incident light stream is in focus at the light modulationunit.

In some embodiments, the image is an n-pixel image with n less than thenumber N of light modulating elements in the light modulation unit, andthe number m of spatial patterns in the sequence of spatial patterns isless than n. The compression ratio m/n may be different in differentembodiments. For example, in different sets of embodiments, thecompression ratio m/n may, respectively be less than or equal to 0.9,less than or equal to 0.8, less than or equal to 0.7, less than or equalto 0.6, less than or equal to 0.5, less than or equal to 0.4, less thanor equal to 0.3, less than or equal to 0.2, less than or equal to 0.1,less than or equal to 0.05, less than or equal to 0.025.

Furthermore, the ratio n/N may be different in different embodiments.For example, in different sets of embodiments, the ratio n/N may be,respectively, less than or equal to ½, less than or equal to ¼, lessthan or equal to ⅛, less than or equal to 1/16, less than or equal to1/32, less than or equal to 1/64, less than or equal to 1/128, less thanor equal to 1/256, less than or equal to 1/512, less than or equal to1/1024, less than or equal to 1/2048, less than or equal to 1/4096, lessthan or equal to 2⁻¹³, less than or equal to 2⁻¹⁴, less than or equal to2⁻¹⁵.

In some embodiments, the method 1200 may also include changing anoptical distance between an optical subsystem of the CI device (e.g.,optical subsystem 105) and the light modulation unit based on the focusinformation, e.g., as variously described above.

In some embodiments, the method 1200 may also include changing a focussetting of the optical subsystem based on the focus information, e.g.,as variously described above.

Display the Reconstructed Image and Let User Control Focus Adjustment

In one set of embodiments, a method 1300 for focusing a compressiveimaging (CI) device may include the actions shown in FIG. 13.(Furthermore, method 1300 may include any subset of the actions,embodiments and features described above in connection with methods 600through 1200.)

Action 1310 includes modulating an incident light stream with a sequenceof spatial patterns to obtain a modulated light stream, e.g., asvariously described above. The modulation is performed by a lightmodulation unit of the CI device (e.g., by the light modulation unit 110as described above).

Action 1315 includes generating an electrical signal representingintensity of the modulated light stream as a function of time, where theelectrical signal is generated by a light sensing device of the CIdevice (e.g., by the light sensing device 130 as described above).

Action 1320 includes acquiring samples of the electrical signal, e.g.,as variously described above. The acquired samples include at least onesample for each of the spatial patterns.

Action 1325 includes constructing an image using the sequence of spatialpatterns and the acquired samples, e.g., using any of the signalreconstruction methods known in the field of compressed sensing.

Action 1330 includes displaying the image on a display (e.g., a displayof the CI device).

Action 1335 includes changing a focus setting of the CI device based onuser input.

In some embodiments, the focus setting of the CI device is an opticaldistance between an optical subsystem of the CI device (e.g., opticalsubsystem 105) and the light modulation unit. In some embodiments, thefocus setting of the CI device is a focus setting of the opticalsubsystem.

In some embodiments, the method 1300 may also include performing actions1310 through 1335 one or more additional times.

Minimizing Spillover to Achieve Focus on Sensor Array

In one set of embodiments, a method for focusing a compressive imaging(CI) device may include the actions shown in FIG. 14A. (Furthermore,method 1400 may include any subset of the actions, embodiments andfeatures described above in connection with system 100, systemrealization 200, and methods 600 through 1300.)

Action 1410 includes modulating an incident light stream withcalibration patterns to obtain a modulated light stream. The modulationis performed by a light modulation unit of the CI device (e.g., thelight modulation unit 110 as described above). The light modulation unitincludes an array of light modulating elements.

At 1415, an array of light sensing elements of the CI device receivesthe modulated light stream. (The light sensing array may be as variouslydescribed above, or as variously described in U.S. patent applicationSer. No. 13/197,304.) The light sensing elements receive respectivespatial portions of the modulated light stream and generate respectiveelectrical signals. Each electrical signal represents intensity of therespective spatial portion as a function of time. The light sensingelements correspond to respective non-overlapping regions within thearray of light modulating elements. Each of the calibration patternsspecifies that the light modulating elements inside a respective one ofthe regions are to be set to an OFF state and specifies that at least asubset of the light modulating elements outside the respective regionare to be set to an ON state. For example, in the context of FIG. 2I, acalibration pattern corresponding to the top left quadrant of the lightmodulation unit might turn OFF the light modulating elements in the topleft quadrant, and turn ON all (or at least some) of the lightmodulating elements in the remaining three quadrants. The number ofcalibrations patterns may be less than or equal to the number of lightsensing elements in the array of light sensing elements.

The ON state or maximum transfer state of a light modulating element isthe state in which the ratio of the intensity of the light portionoutputted from the element (onto the path that leads to the lightsensing array) to the intensity of the light portion incident upon theelement is maximized, within the range of control supported by theelement. (For example, for an element having controllable transmittance,such as an LCD shutter, the maximum transfer state is the state ofmaximum transparency. As another example, for a reflective element suchas a micromirror in a DMD, the maximal transfer state is the orientationstate that reflects the received light portion onto the sensing path115.)

The OFF state or minimum transfer state of a light modulating element isthe state in which the ratio of the intensity of the light portionoutputted from the element (onto the path that leads to the lightsensing array) to the intensity of the light portion incident upon theelement is minimized, within the range of control supported by theelement. (For example, for an element having controllable transmittance,such as an LCD shutter, the minimum transfer state is the state ofminimum transparency, i.e., maximum blockage of light. As anotherexample, for a reflective element such as a micromirror in a DMD, theminimum transfer state is the orientation state that reflects thereceived light portion away from the sensing path 115.)

As indicated at 1420, for each of the calibration patterns, a respectivegroup of samples is acquired from the array of light sensing elements,e.g., as variously described above, and/or, as variously described inU.S. patent application Ser. No. 13/197,304. Each of the sample groupsincludes at least one sample of each of the electrical signals, i.e.,one sample from each of the light sensing elements). (In someembodiments, two or more groups of samples may be acquired percalibration pattern. Corresponding samples in the two or more groups areaveraged to obtain a refined sample group per calibration pattern, withthe refined sample group including one refined sample per electricalsignal. In these embodiments, the refined sample groups will substitutefor the sample groups described in the operations below.)

Action 1425 includes determining spillover values correspondingrespectively to the calibration patterns. Each of the spillover valuesis determined based on the sample group corresponding to the respectivecalibration pattern. Each of the spillover values indicates an extent towhich modulated light from outside the respective region of the array oflight modulating elements reaches the light sensing elementcorresponding to the respective region. For example, a first of thespillover values indicates an extent to which modulated light from lightmodulating elements outside a first of the regions (of the lightmodulating array) reaches a first of the light sensing elements. In thecontext of FIG. 2I, the first spillover value may correspond to a firstcalibration pattern that turns ON only the second, third and fourthquadrants of the light modulating array. The first spillover value maybe determined simply by extracting the sample from the sample group thatcorresponds to the light sensing device of the first quadrant. Thatsample indicates how much light has spilled over from the second, thirdand fourth quadrants.

Action 1430 includes computing a composite spillover value for the arrayof light sensing elements based on the spillover values. In someembodiments, the composite spillover value may be computed based on anaverage (or a maximum or a statistic) of the spillover values.

In some embodiments, method 1400 may also include directing an actuatorto change an optical distance between the light modulation unit and thearray of light sensing elements based on data including the compositespillover value. The optical distance may be changed, e.g., bytranslating the array of light sensing elements along the axis of themodulated light stream.

In some embodiments, the method 1400 may include repeatedly performing aset of operations, wherein the set of operations includes changing theoptical distance between the light modulation unit and the array oflight sensing elements, and performing actions 1410 through 1430. Theset of operations may be repeated until the composite spillover value isminimized or becomes less than a threshold value. The minimizing stateof the optical distance may be interpreted as a state of optimal (oracceptable) focus. In some embodiments, having achieved thisminimization, another one of the focus methods described herein may beused to further improve the state of focus.

In some embodiments, the method 1400 may also include directing anactuator to change an angle of orientation of the array of lightmodulating elements based on data including the composite spillovervalue.

In some embodiments, the method 1400 may include repeatedly performing aset of operations, where the set of operations includes changing theangle of orientation of the array of light sensing elements, andperforming actions 1410 through 1430. The set of operations may berepeated until the composite spillover value is minimized or becomesless than a threshold value.

In some embodiments, each calibration pattern specifies that the lightmodulating elements within the respective region are to take the OFFstate and that the light modulating elements outside the respectiveregion are to take the ON state.

Generic Step of an Adaptive Algorithm

In some embodiments, the method 1400 may also include: directing anactuator to change the optical distance between the light modulationunit and the array of light sensing elements according to a firstdisplacement value; performing actions 1410 through 1430 a second time,in order to obtain an additional composite spillover value (after theactuator has changed the optical distance); and computing a seconddisplacement value for the optical distance based on data including thecomposite spillover value, the additional composite spillover value andthe first displacement value. (In some embodiments, the displacementvalues may vary in direction and magnitude. In other embodiments, thedisplacement values may be unit steps, varying only in direction.) Theactuator may then be directed to change the optical distance accordingto the second displacement value.

In some embodiments, the method 1440 may also include: directing anactuator to change the angle of orientation of the array of lightsensing elements according to a first displacement value; performingactions 1410 through 1430 a second time, in order to obtain anadditional composite spillover value (after the actuator has changed theangle of orientation); and computing a second displacement value for theangular orientation based on data including the composite spillovervalue, the additional composite spillover value and the firstdisplacement value. (In some embodiments, the actuator allowsvariable-sized steps. In other embodiments, the actuator may allow onlyunit-sized steps.) The actuator may then be directed to change the angleof orientation of the array of light sensing elements according to thesecond displacement value.

Batch Algorithm

In some embodiments, the method 1400 may include: directing an actuatorto change the optical distance between the light modulation unit and thearray of light sensing elements through a range of distances; performingactions 1410 through 1430 for each of a plurality of optical distanceswithin the range in order to obtain a corresponding plurality ofcomposite spillover values; and determining an optimal value for theoptical distance based on an analysis of data including the plurality ofcomposite spillover values (and in some embodiments, also the pluralityof optical distances). The actuator may then be directed to change thedistance to the optimal value. The optimal value may be determined,e.g., by minimizing the function of optical distance given by thecomposite spillover values and the plurality of optical distances.

In some embodiments, the method 1400 may include: directing an actuatorto change the orientation angle of the array of light sensing elementsthrough a range of angles; performing actions 1410 through 1430 for eachof a plurality of angles within the range in order to obtain acorresponding plurality of composite spillover values; and determiningan optimal value for the angle based on an analysis of data includingthe plurality of composite spillover values (and in some embodiments,also the plurality of angles). The actuator may then be directed tochange the angular orientation to the optimal value. The optimal valuemay be determined, e.g., by minimizing the function of angle given bythe composite spillover values and the plurality of angles).

In some embodiments, the number of the calibration patterns is four. Inone embodiment, the four regions of the light modulating array thatcorrespond to the four calibration patterns are located at (or near) thefour corners of the light modulating array.

In some embodiments, the number of regions of the light modulating array(which is the same as the number of light sensing elements) is largeenough so that not all the regions are in contact with the boundary ofthe light modulating array. For example, with a 4×4 matrix of regions,there is a 2×2 submatrix of regions in the interior, not contacting anyof the boundary of the light modulating array. In some embodiments, theregions corresponding to the calibration patterns are interior regionsof the light modulating array, i.e., regions not touching the boundaryof the light modulating array.

FIG. 14B illustrates one embodiment of the calibration patterns in thecase where there are four light sensing elements arranged in a 2×2array, e.g., as shown in FIG. 2I. Each of the calibration patterns P1-P4is turned OFF in a respective quadrant of the light modulating array,and is turned ON along portions of the remaining quadrants that boundthe respective quadrant. This principle of pattern creation generalizesto any number of regions.

Method 1500: Single Calibration Pattern—Single Spillover Value

In one set of embodiments, a method 1500 may include the actions shownin FIG. 15. (Furthermore, method 1500 may include any subset of theactions, embodiments and features described above in connection withsystem 100, system realization 200 and methods 600 through 1400.)

Action 1510 includes modulating an incident light stream with acalibration pattern to obtain a modulated light stream. The action ofmodulating the incident light stream is performed by a light modulationunit of the CI device (e.g., by the light modulation unit 110 asdescribed above). The light modulation unit includes an array of lightmodulating elements.

At 1515, an array of light sensing elements of the CI device receivingthe modulated light stream, where the light sensing elements receiverespective spatial portions of the modulated light stream and generaterespective electrical signals, where each electrical signal representsintensity of the respective spatial portion as a function of time, wherethe light sensing elements correspond to respective non-overlappingregions within the array of light modulating elements, where thecalibration pattern specifies that the light modulating elements insidea particular one of the regions are to be set to an off state andspecifies that at least a subset of the light modulating elementsoutside the particular region are to be set to an on state. Theparticular region corresponds to a particular one of the light sensingelements.

Action 1520 includes acquiring samples of the electrical signalgenerated by the particular light sensing element in response to saidmodulating the incident light stream with the calibration pattern. Eachof the samples indicates an extent to which modulated light from outsidethe particular region of the array of light modulating elements reachesthe particular light sensing element. Thus, each of the samples may beinterpreted as a measure of spillover S. In some embodiments, thesamples or a subset of the samples may be averaged, and the average maybe used as the measure of spillover S.

Any subset of the embodiments described above in connection with method1400 may be incorporated into method 1500, with the provision that the“composite spillover” of method 1400 is replaced by the spillovermeasure S mentioned above.

Alternative Spillover Measurement

In one set of embodiments, a method for focusing a compressive imaging(CI) device may include the following actions. (Any subset of theembodiments described above in connection with method 1400 may beincorporated into this method. Furthermore, any subset of the actions,embodiments, features described in this patent may be incorporated intothis method.)

An incident light stream may be modulated with calibration patterns toobtain a modulated light stream. The action of modulating is performedby a light modulation unit of the CI device (e.g., the light modulationunit 110 as described above). The light modulation unit includes anarray of light modulating elements.

An array of light sensing elements of the CI device receives themodulated light stream. Each of the light sensing elements receives arespective spatial portion of the modulated light stream and generates arespective electrical signal representing intensity of the respectivespatial portion as a function of time. The calibration patternscorrespond to respective regions of the array of light modulatingelements and correspond to respective ones of the light sensingelements. Each of the calibration patterns specifies that at least asubset of the light modulating elements inside the respective region areto be set to an on state and specifies that the light modulatingelements outside the respective region are to be set to an off state.The regions are preferably non-overlapping regions of the lightmodulating array. The number of calibrations patterns may be less thanor equal to the number of light sensing elements in the array of lightsensing elements.

For each of the calibration patterns, a respective group of samples isacquired from the array of light sensing elements. Each of the samplegroups includes at least one sample of each of the electrical signals,i.e., one sample from each of the light sensing elements. (In someembodiments, a two are more sample groups are acquired per calibrationpattern, in which case corresponds samples in the two or more samplegroups may be averaged to obtain a refined sample group, the refinedsample groups containing one refined (averaged) sample per light sensingdevice.)

Spillover values are computed, where the spillover values correspondrespectively to the calibration patterns. Each of the spillover valuesis computed based on the sample group corresponding to the respectivecalibration pattern. Each of the spillover values indicates an extent towhich modulated light from the respective region of the array of lightmodulating elements reaches light sensing elements of the array otherthan the respective light sensing element. For example, a first of thespillover values indicates an extent to which modulated light from afirst one of the regions of the light modulating array reaches lightmodulating elements other than a first of the light sensing elements. Inthe context of FIG. 2I, the first spillover value may correspond to acalibration pattern that turns on light modulating element only in thefirst quadrant. The first spillover value may be computed by summing thesamples of the sample group corresponding to the light sensing devicesfor quadrants two, three and four.

A composite spillover value may be computed for the array of lightsensing elements based on the spillover values. In some embodiments, thecomposite spillover value may be computed based on an average (or amaximum or a statistic) of the spillover values.

In some embodiments, each of the calibration patterns specifies that allthe light modulating elements within the respective region are to takean ON state.

In one set of embodiments, a method for focusing a compressive imaging(CI) device may include the following actions. (Any subset of theembodiments described above in connection with method 1400 may beincorporated into this method. Furthermore, any subset of the actions,embodiments, features described in this patent may be incorporated intothis method.)

An incident light stream is modulated with a calibration pattern toobtain a modulated light stream. The operation of modulating isperformed by a light modulation unit of the CI device (e.g., by thelight modulation unit 110 as described above). The light modulation unitincludes an array of light modulating elements.

An array of light sensing elements of the CI device receives themodulated light stream. Each of the light sensing elements receives arespective spatial portion of the modulated light stream and generates arespective electrical signal representing intensity of the respectivespatial portion as a function of time. The calibration patterncorresponds to a region of the array of light modulating elements and toone of the light sensing elements. Furthermore, the calibration patternspecifies that at least a subset of the light modulating elements withinthe region are to be set to the on state, and specifies that the lightmodulating elements outside the region are to be set to the off state.

A group of samples corresponding to the calibration pattern is acquired,wherein the sample group includes at least one sample of each of theelectrical signals.

A spillover value for the calibration pattern is computed. The spillovervalue is computed based on the sample group and indicates an extent towhich modulated light (i.e., light of the modulated light stream) fromthe region of the array of light modulating elements reaches lightsensing elements other than the corresponding light sensing element(i.e., the element that corresponds to the region and the calibrationpattern).

Measuring Noise in Reconstructed Sub-Images Corresponding to RespectiveLight Sensing Elements

In one set of embodiments, a method 1640 for determining focus-relatedinformation for a compressive imaging (CI) device may include theactions shown in FIG. 16A. (Furthermore, method 1640 may include anysubset of the actions, embodiments and features described herein, andespecially, in connection with methods 600 through 1500.)

Action 1642 includes modulating an incident light stream with a sequenceof spatial patterns to obtain a modulated light stream. The modulationis performed by a light modulation unit of the CI device. The lightmodulation unit includes an array of light modulating elements. Thelight sensing elements correspond to respective non-overlapping regionson the array of light modulating elements (e.g., as suggested in FIG.2I). The spatial patterns may each have the same number of elements asthe light modulating array.

At 1644, the array of light sensing elements of the CI device receivingthe modulated light stream, where the light sensing elements receiverespective spatial portions of the modulated light stream and generaterespective electrical signals, where each of the electrical signalsrepresents intensity of the respective spatial portion as a function oftime.

Action 1646 includes acquiring measurements from the array of lightsensing elements, where the measurements include sample sets thatcorrespond respectively to the light sensing elements, where each sampleset includes samples of the electrical signal generated by therespective light sensing element, with the samples including at leastone (e.g., exactly one) sample for each of the spatial patterns;

Action 1648 includes computing a focus indicator value based on two ormore noise values for two or more respective ones of the light sensingdevices. The focus indicator value indicates an extent to which themodulated light stream is in focus at the sensor array. The focusindicator value may be determined by computing an average (or a maximumor a statistic) of the two or more noise values.

Each of the two or more noise values may be computed by: constructing arespective sub-image based on the sample set of the respective lightsensing element and also on a restriction of the spatial patterns to theregion (of the light modulating array) corresponding to the respectivelight sensing element; and computing an amount of noise present in therespective sub-image. (Here, the term “restriction” of a spatial patternto a region means restriction of the domain of the spatial pattern sothat the restricted pattern exists only on the region.)

For example, in the context of FIG. 2I, a first noise value may becomputed by constructing a first sub-image based on the samples from thefirst light sensing element and the restriction of the spatial patternsto the first quadrant of the light modulating array (i.e., the quadrantthat is supposed to supply the first light sensing element), and thencomputing an amount of noise present in the first sub-image.

The noise value for a sub-image may be computed by evaluating a blindnoise measure on the sub-image, “blind” meaning that the measure isevaluated without using a reference sub-image (an ideal sub-image ofwhich the constructed sub-image is an approximation). Any of a widevariety of blind noise measures known in the field of image processingmay be used. Alternatively, if a reference image is available, the noisevalue for the constructed sub-image may be determined, e.g., bycomputing a standard deviation (or a variance, or an average of absolutevalues, or a sum of absolute values) of pixel-wise differences betweenthe constructed sub-image and the corresponding sub-image of thereference image. In some embodiments, the noise value may be determinedbased on averaging or summing values f(|d_(i,j)|), where {d_(i,j)} arethe pixel differences and f is any increasing function. In somesituations, a user may print and display a reference image and set upthe reference image at a target distance from the CI device prior to theexecution of method 1640.

Because each sub-image is computed based on a restriction of spatialpatterns to the respective region, it has a pixel count equal to theelement count of the region, which is smaller than the element count ofthe whole light modulating array.

In some embodiments, the method 1640 may also include: determining adisplacement value for the array of light sensing elements based on dataincluding the focus indicator value (e.g., a displacement for theposition of the array along an axis of the modulated light stream); anddirecting the array of light sensing elements to be translated accordingto the displacement value. The light sensing array may be mounted on atranslation stage that admits translation along an axis of the modulatedlight stream so as to increase and decrease the optical distance betweenthe light modulation unit and the light sensing array.

In some embodiments, method 1640 may include: determining an angulardisplacement value for the array of light sensing elements based on dataincluding the focus indicator value; and directing the array of lightsensing elements to be rotated according to the angular displacementvalue. The light sensing array may be mounted on a rotation stage thatadmits one or more axes of rotation. If the light sensing array isinitially not oriented correctly with respect to the modulated lightstream, knowledge of the focus indicator value (and perhaps also one ormore previously-determined focus indicator values, corresponding to oneor more previous states of the orientation angle) may be used tocorrectly orient the light sensing array, e.g., so that the plane of thearray is perpendicular to the axis of the modulated light stream.

In some embodiments, the method 1640 may also include changing anoptical distance between the light modulation unit 110 and the array oflight sensing elements, e.g., by translating the array of light sensingelements, as variously describe herein. The action of changing theoptical distance and actions 1642 through 1648 may be performed one ormore times until the focus indicator value is optimized (e.g.,minimized).

In some embodiments, the method 1640 may also include changing anangular orientation of the array of light sensing elements. The actionsof changing the angular orientation and actions 1642 through 1648 may beperformed one or more times until the focus indicator value is optimized(e.g., minimized).

Measuring Noise in a Single Reconstructed Sub-Image

In some embodiments, a method for determining focus-related informationfor a compressive imaging (CI) device may include the following actions.(Furthermore, this method may include any subset of the actions,features and embodiments disclosed herein, e.g., as subset of theembodiments described in connection with method 1640.)

An incident light stream may be modulated with a sequence of spatialpatterns to obtain a modulated light stream. The modulating is performedby a light modulation unit of the CI device, where the light modulationunit includes an array of light modulating elements.

An array of light sensing elements of the CI device receives themodulated light stream, where the light sensing elements receiverespective spatial portions of the modulated light stream and generaterespective electrical signals. Each of the electrical signals representsintensity of the respective spatial portion as a function of time. Thelight sensing elements correspond to respective non-overlapping regionsof the array of light modulating elements. In other words, each lightsensing element is supposed to receive light from the respective regionof the light sensing array. However, until proper focus is achieved,significant amounts of light from regions other than the respectiveregion may be received at the light sensing element.

A set of samples is acquired from a particular one of the light sensingelements. The sample set includes samples of the electrical signalgenerated by the particular light sensing element, with the samplesincluding at least one sample (e.g., exactly one sample) for each of thespatial patterns.

A focus indicator value may be computed for the particular light sensingdevice. The focus indicator value indicates an extent to which themodulated light stream is in focus at the sensor array. The focusindicator value may be computed by: constructing a sub-image based onthe sample set and on a restriction of the spatial patterns to theregion of the array of light modulating elements that corresponds to thefirst light sensing element; and computing an amount of noise present inthe sub-image. The noise amount may be taken as the focus indicatorvalue.

The amount of noise may be computed in a blind fashion and/or using areference, as variously described above.

Measuring Spillover in Reconstructed Images Corresponding to RespectiveLight Sensing Elements

In one set of embodiments, a method 1660 for determining focus-relatedinformation for a compressive imaging (CI) device may include theactions shown in FIG. 16B. (Furthermore, method 1660 may include anysubset of the actions, embodiments and features described herein, e.g.,any subset of the actions, embodiments and features described inconnection with methods 600 through 1640.)

Action 1662 includes modulating an incident light stream with a sequenceof spatial patterns to obtain a modulated light stream, e.g., asvariously described above. The action of modulating is performed by alight modulation unit of the CI device. The light modulation unitincludes an array of light modulating elements.

Action 1664 includes an array of light sensing elements of the CI devicereceiving the modulated light stream, where the light sensing elementsreceive respective spatial portions of the modulated light stream andgenerate respective electrical signals, where each of the electricalsignals represents intensity of the respective spatial portion as afunction of time, where the light sensing elements correspond torespective non-overlapping regions on the array of light modulatingelements.

Action 1666 includes acquiring measurements from the array of lightsensing elements, where the measurements include sample sets thatcorrespond respectively to the light sensing elements. Each sample setincludes samples of the electrical signal generated by the respectivelight sensing element, with the samples including at least one (e.g.,exactly one) sample for each of the spatial patterns.

Action 1668 includes computing a focus indicator value based on two ormore spillover values for two or more respective ones of the lightsensing devices, wherein the focus indicator value indicates an extentto which the modulated light stream is in focus at the sensor array.Each of the two or more spillover values is computed by: constructing arespective image based on the sample set of the respective light sensingelement and also on the spatial patterns; and summing pixels in theimage outside the region corresponding to the respective light sensingelement.

Because each image is constructed using the spatial patterns withoutrestriction of spatial domain, the pixel dimensions of the image are thesame as the element dimensions of the light modulating array. (However,in alternative embodiments, the spatial patterns used to construct agiven image may be restricted at least partially. For example, one mightconstruct an image for a given light sensing device using patternsrestricted to the corresponding region plus a boundary zone around thecorresponding region.)

For example, in the context of FIG. 2I, a first spillover value for afirst light sensing device may be computed by constructing an imagebased on the samples acquired by the first light sensing device, e.g.,an image whose pixel dimensions agree with the element dimensions of thelight sensing array. FIG. 16C shows such an image, where the first lightsensing device corresponds to (is supposed to receive light from) thetop right quadrant of the light modulating array. Portions of the imageoutside the top-right image quadrant are due to light spillover from theother quadrants of the light modulating array to the first light sensingdevice. Thus, the first spillover value may be determined by computing asum of the pixels in the other three image quadrants (top left, bottomleft and bottom right).

Any of various mechanisms for adjusting focus described herein may beincluded in this method.

Measuring Spillover in a Single Reconstructed Image Corresponding to aSingle Light Sensing Element

In one set of embodiments, a method for determining focus-relatedinformation for a compressive imaging (CI) device may include thefollowing actions. (Furthermore, this method may include any subset ofthe actions, features and embodiments disclosed herein, e.g., as subsetof the embodiments described in connection with methods 60 through1660.)

An incident light stream may be modulated with a sequence of spatialpatterns to obtain a modulated light stream. The action of modulating isperformed by a light modulation unit of the CI device. The lightmodulation unit includes an array of light modulating elements.

An array of light sensing elements of the CI device receives themodulated light stream, where the light sensing elements receiverespective spatial portions of the modulated light stream and generaterespective electrical signals. Each of the electrical signals representsintensity of the respective spatial portion as a function of time. Thelight sensing elements correspond to respective non-overlapping regionson the array of light modulating elements, e.g., as variously describedabove.

Measurements are acquired from the array of light sensing elements. Themeasurements include sample sets that correspond respectively to thelight sensing elements. Each sample set includes samples of theelectrical signal generated by the respective light sensing element,with the samples including at least one (e.g., exactly one) sample foreach of the spatial patterns.

A spillover value is computed for a particular one of the light sensingdevices. The spillover value indicates an extent to which the modulatedlight stream is in focus at the sensor array (or at least at theparticular light sensing device). The spillover value may be computedby: constructing an image based on the sample set corresponding to theparticular light sensing element and also on the spatial patterns (sothe pixel dimensions of the image are the same pixel dimensions of eachspatial pattern); and summing pixels in the image outside the regioncorresponding to the particular light sensing element.

Focusing CI Device Using High Frequency Spatial Patterns

In one set of embodiments, a method 1700 for focusing a compressiveimaging (CI) device may include the actions shown in FIG. 17.(Furthermore, method 1700 may include any subset of the actions,embodiments and features described in this patent.)

Action 1710 includes supplying a sequence of high-frequency spatialpatterns to a light modulation unit of the CI device. The lightmodulation unit includes an array of light modulating elements, e.g., asvariously described above. Each of the high-frequency spatial patternsis a spatial pattern whose spatial AC component includes only spatialfrequencies greater than or equal to a cutoff frequency. In general, aspatial pattern—whether high-frequency or not—may be interpreted as anspatial array of scalar multipliers. See the above discussion of spatialpatterns in connection with system 100. The AC component of a spatialpattern is the spatial pattern minus its DC value. The DC value is theaverage value of the spatial pattern. In some embodiments, each of thehigh-frequency spatial patterns is configured to have the same DC value.

Action 1715 includes modulating an incident light stream with thesequence of high-frequency spatial patterns to produce a modulated lightstream. The modulation is performed a light modulation unit of the CIdevice (e.g., by the light modulation unit 110).

At 1720, a light sensing device of the CI device (e.g., the lightsensing device 130 described above) receives the modulated light streamand acquires a sequence of samples representing an intensity of themodulated light stream as a function of time. The sequence of samplesmay include at least one sample for each of the high-frequency spatialpatterns.

Action 1725 includes computing a focus indicator value based on thesequence of samples. The focus indicator value indicates an extent towhich the incident light stream is in focus at the array of lightmodulating elements.

In some embodiments, each of the high-frequency spatial patternscorresponds to spatial sinusoid whose spatial frequency is greater thanor equal to the cutoff frequency. For example, each high-frequencyspatial pattern may be a DC-shifted sinusoid.

In some embodiments, the cutoff-frequency is programmable.

In some embodiments, a known image K may be printed (or otherwisedisplayed) and set up at a desired distance from the CI device. The CIdevice may be pointed at the printed image. Then method 1700, in any ofits various embodiments, may be performed to focus the CI device on theprinted image. The image K may be designed so that it has significanthigh-frequency (small wavelength) content. For example, the image K mayinclude series of vertical and/or horizontal lines that are sharp anddensely spaced. (Any of the various method embodiments described hereinmay likewise use a print out of a known image to facilitatedetermination of proper focus, e.g., during calibration of the CIdevice.)

In some embodiments, the CI device also includes an optical subsystem(e.g., the optical subsystem 105) that receives and operates on theincident light stream prior to the incident light stream arriving at thelight modulation unit. Thus, the method 1700 may include changing afocus setting of the optical subsystem, e.g., based on data includingthe focus indicator value.

In some embodiments, the method 1700 may include repeatedly performing aset of operations, where the set of operations includes: changing afocus setting of the optical subsystem; and performing actions 1710through 1725. The set of operations may be repeated until the focusindicator value becomes less than a threshold value.

In some embodiments, method 1700 may include changing an opticaldistance between the light modulation unit and the optical subsystem,e.g., based on data including the focus information. As variouslydescribed above, the optical distance may be changed by translating thelight modulation unit, or alternatively, by translating the opticalsubsystem.

In some embodiments, the method 1700 may include repeatedly performing aset of operations, where the set of operations includes: changing anoptical distance between the light modulation unit and the opticalsubsystem; and performing actions 1710 through 1725. The set ofoperations may be repeated until the focus indicator value becomes lessthan a threshold value.

In some embodiments, the method 1700 may also include: directing anactuator to change an optical distance between the optical subsystem andthe light modulation unit according to a first displacement value;performing actions 1710 through 1725 a second time, to obtain anadditional focus indicator value for an additional sequence of samples(e.g., using the same sequence of high-frequency spatial patterns); andcomputing a second displacement value for the optical distance based ondata including the focus indicator value for the sequence of samples,the focus indicator value for the additional sequence of samples and thefirst displacement value. The actuator may then be directed to changethe optical distance based on the second displacement value.

In some embodiments, the method 1700 may also include: directing anactuator to change a focus setting of the optical subsystem according toa first displacement value; performing actions 1710 through 1725 asecond time, to obtain an additional focus indicator value for anadditional sequence of samples (e.g., using the same sequence ofhigh-frequency spatial patterns); and computing a second displacementvalue for the focus setting based on data including the focus indicatorvalue for the sequence of samples, the focus indicator value for theadditional sequence of samples and the first displacement value. Theactuator may then be directed to change the focus setting of the opticalsubsystem according to the second displacement value.

In some embodiments, the action 1725 of computing the focus indicatorvalue includes: removing a temporal DC component of the sequence ofsamples to obtain an AC sequence of samples; and computing an averagemagnitude of the AC sequence of samples. The average magnitude may beused as the focus indicator value. The temporal DC component may beremoved, e.g., by highpass filtering the sequence of samples, or, bynumerically computing the average value of the sequence and thensubtracting the average value of each sample of the sequence.

Compressive Imaging Device with Time-of-Flight Determining Device

In one set of embodiments, a system 1800 may be configured as shown inFIG. 18. System 1800 may include the light modulation unit 110M, thelight sensing device 130 and processing unit 150 as described above, andmay also include a time-of-flight determining device (TDD) 1810.

The light modulation unit 110M is configured to receive an incidentlight stream L from an optical input path IOP, e.g., as variouslydescribed above, especially in connection with system realization 200.(See, e.g., FIGS. 3A through 5C.) The incident light stream L isindicated as a bundle of rays entering the input optical path.Thereafter, it is indicated only by representative rays to avoid clutterin the diagram.

The light modulation unit 110M includes an array of mirrors configuredto receive respective portions of the incident light stream. Each of themirrors is configured to (a) reflect the respective portion of theincident light stream L onto a first optical path P₁ when the mirror isin a first orientation state and (b) reflect the respective portion ofthe incident light stream L onto a second optical path P₂ when themirror is in a second orientation state. Each mirror includes at leasttwo orientation states. (In some embodiments, each mirror has more thantwo orientation states.)

The light sensing device 130 is configured to receive a first lightstream S₁ comprising the portions of the incident light stream L thatare reflected onto the first optical path P₁ by mirrors in the firstorientation state. Furthermore, the light sensing device is configuredto generate a sequence of samples representing intensity of the firstlight stream S₁ as function of time.

The time-of-flight determining device (TDD) 1810 is configured to: (a)transmit a light pulse onto the second optical path P₂ so that the lightpulse is reflected onto the input optical path IOP and out of the systemby mirrors that are in the second orientation state, (b) receive asecond light stream S₂ comprising the portions of the incident lightstream L that are reflected onto the second optical path by the mirrorsin the second orientation state, (c) detect a reflected pulse occurringin the second light stream S₂, and (d) determine a time of flightbetween the transmission of the light pulse and detection of thereflected pulse. Thus, the second optical path P₂ carries both incomingand outgoing light signals. Furthermore, the input output path duels asan output path for the transmitted light pulse.

The processing unit 150 may be configured to determine a range betweenthe system (e.g., the light modulation unit 110M) to an external objectbased on the time of flight. For example, the range may be determinedaccording to the relation R=cΔt/2, where c is the speed of light and Δtis the time of flight.

In some embodiments, the light used by the TDD 1810 for transmitting thelight pulse may reside in (or be restricted to) a wavelength range thatis outside the wavelength range sensed by the light sensing device LSD130.

In some embodiments, the light used by the TDD 1810 for transmitting thelight pulse may be monochromatic light of a single wavelength, e.g.,laser light.

In some embodiments, the light used by the TDD 1810 for transmitting thelight pulse may be light in the visible range or in the UV range or inthe IR range.

In some embodiments, the beam used by the TDD 1810 to transmit the lightpulse is configured so that its central axis is pointed at the center ofthe array of light modulating elements.

The diameter of the beam used by the TDD to transmit the light pulse maybe different in different embodiments, e.g., in one embodiment, beinglarge enough so that the beam covers the whole (or the vast majority) ofthe light modulating array, in another embodiment, being so small thatthe beam covers only a few light modulating elements, and in otherembodiments, any state between those extremes.

In some embodiments, the system 1800 also includes the optical subsystem105 described above. The optical subsystem 105 is positioned along theinput optical path IOP and configured to operate on the incident lightstream before it is provided to the light modulation unit 110M. In someembodiments, the optical subsystem 105 includes a camera lens.

In some embodiments, the system 1800 also includes a first actuatorconfigured to adjust a first optical distance d₁ between the opticalsubsystem 105 and the light modulation unit 110M (e.g., by translatingthe light modulation unit). The processing unit may be configured tocompute an adjustment value Δd₁ for the first optical distance based onthe determined range and to direct the first actuator to adjust thefirst optical distance according to the adjustment value Δd₁.Alternatively, the processing unit may compute an optimal value for thefirst optical distance (based on the determined range) and direct thefirst actuator to set the first optical distance to the optimal value.

In some embodiments, system 1800 also includes a second actuatorconfigured to adjust a second optical distance d₂ between the lightmodulation unit 100M and the light sensing device 130. The processingunit 150 may be configured to compute an adjustment value Δd₂ for thesecond optical distance based on the adjustment value Δd₁ (or based onthe determined range R) and to direct the second actuator to adjust thesecond optical distance according to the adjustment value Δd₂.

In some embodiments, the system 1800 may also include a third actuatorconfigured to adjust an angular orientation of the light sensing device.The processing unit 150 may be configured to compute an angle adjustmentvalue Δθ for the angular orientation based on the first adjustment valueΔd₁ (or based on the determined range) and to direct the second actuatorto adjust the angular orientation according to the angle adjustmentvalue.

In some embodiments, the system 1800 may also include an actuatorconfigured to adjust a focus setting of the optical subsystem 105. Theprocessing unit 150 may be configured to direct the actuator to adjustsaid focus setting based on the determined range.

In some embodiments, the TDD 1810 includes a transmitter 1910, adetector 1915, and a timing subsystem 1920, e.g., as shown in FIG. 19.The transmitter is configured to generate the light pulse. The detectoris configured to receive the second light stream and to detect thereflected pulse occurring in the second light stream. The timingsubsystem is configured to determine the time of flight between thetransmission of the light pulse and the detection of the reflectedpulse. In some embodiments, the TDD may also include an amplifier 1917and a beam splitter 1925. In some embodiments, the transmitter 1910includes a laser light source.

In some embodiments, the TDD 1810 is a laser range finder. Laser rangefinders are widely available from a variety of manufacturers andvendors.

In some embodiments, the system 1800 may also include the control unit120 as described above. The control unit is configured to drive thearray of mirrors through a sequence of spatial patterns. The TDD 1810may be is configured to transmit the light pulse and detect thereflected pulse within a period of time corresponding to a single one ofthe spatial patterns. The TDD may receive a pattern clock signal (e.g.,from control unit 120 or processing unit 150) that controls thetransitions from one pattern to the next. The TDD may detect an activeedge of the pattern clock, wait for a predetermined amount of time toallow transients to decay, and then transmit the light pulse after thanpredetermined wait time has expired.

In some embodiments, the sequence of samples corresponds to M of thespatial patterns, wherein the sequence of samples is usable to constructan N-pixel image, wherein M is less than N.

In some embodiments, the TDD 1810 is configured to determine a Dopplershift between the light pulse and the reflected pulse. The processingunit may be configured to compute a range velocity based on the Dopplershift. The range velocity may be used to continuously (or periodicallyor intermittently) adjust the focus setting of the optical subsystem 105and/or to continuously (or periodically or intermittently) adjust theoptical distance between the optical subsystem and the light modulationunit.

In some embodiments, the light pulse transmission may be repeated anumber of times, and the resulting time-of-flight values may be averagedto determined an improved time of flight value.

In some embodiments, system 1800 may also include the optical subsystem117 described above. The optical subsystem 117 may be configured todeliver (or focus or concentrate) the first light stream S₁ onto a lightsensing surface of the light sensing device 130. Similarly, system 1800may also include an optical subsystem configured to deliver (or focus orconcentrate) the second light stream onto a light sensing surface (orport) of the TDD 1810.

In some embodiments, system 1800 may be configured as shown in FIG. 20A.The mirror 117M and lens 117L are configured to deliver the first lightstream S₁ to the light sensing device 130. The mirror 118M and the lens118L are configured to deliver the second light stream S₂ to the TDD1810 and also to deliver the transmitted light pulse from the TDD to thelight modulation unit 110M. In FIG. 20, the input optical path passesthrough camera lens 105CL.

In some embodiments, system 1800 may include a dual TIR prism 2010 asshown in FIG. 20B. The dual TIR prism 2010 is configured to separate theincident light stream L from the first light stream S₁ and the secondlight stream S₂. (The dual TIR prism 2210 may be the same as or similarto the dual TIR prism 2210 used in system 2200. See the discussion ofsystem 2200 below.) Note that the first light stream S₁ is totallyinternally reflected at partially-internal surface K1, and thus,directed onto the optical path that leads to the light sensing device130. The second light stream S₂ is totally internally reflected atpartially-internal surface K2, and thus, directed onto the optical paththat leads to the TDD 1810. Furthermore, the light pulse transmitted bythe TDD is totally internally reflected at the partially-internalsurface K2, and thus, directed to the light modulation unit 110M, whereit is transmitted out of the system via the mirrors in the OFF state.

Compressive Imaging Device with Range Finder

In one set of embodiments, a system 1800 may be configured as shown inFIG. 21. (Furthermore, system 2100 may include any subset of thefeatures, embodiments and elements described above in connection withsystem 1800, and/or, any subset of the features, embodiments andelements described in this patent.)

The system 2100 may include the light modulation unit 110M and the lightsensing device 130 as described above, and may also include a rangefinder 2110.

The light modulation unit 100M is configured to receive an incidentlight stream L from an optical input path IOP. The light modulation unitincludes an array of mirrors configured to receive respective portionsof the incident light stream. Each of the mirrors is configured to (a)reflect the respective portion of the incident light stream onto a firstoptical path P₁ when the mirror is in a first orientation state and (b)reflect the respective portion of the incident light stream onto asecond optical path P₂ when the mirror is in a second orientation state.

The light sensing device 130 is configured to receive a first lightstream S₁ comprising the portions of the incident light stream that arereflected onto the first optical path P₁ by mirrors in the firstorientation state. The light sensing device is configured to generate asequence of samples representing intensity of the first light stream S₁as function of time.

The range finder 2110 is configured to transmit a light signal onto thesecond optical path P₂ so that the light signal is reflected onto theinput optical path IOP and out of the system by mirrors that are in thesecond orientation state. The range finder receives a second lightstream S₂ comprising the portions of the incident light stream that arereflected onto the second optical path P₂ by the mirrors in the secondorientation state. The second light stream S₂ includes a reflected lightsignal due to reflection of the transmitted light signal from anexternal object. The range finder is configured to determine a range tothe external object based on the transmitted light signal and thereflected light signal.

Note that the term “light signal” is broader in its scope of meaningthan the term “light pulse”. Thus, system 2100 encompasses embodimentsthat transmit light pulses and also embodiments that transmit lightsignals that cannot be characterized as pulses.

The range finder 2110 may be a prior art range finder (e.g., a laserrange finder) or a prior art LIDAR device. (LIDAR is acronym for “LightDetection and Ranging”.) Laser range finders and LIDAR devices are wellknown and widely available.

In some embodiments, the ranger finder 2110 may transmit the lightsignal at a wavelength (or in a wavelength band) that is outside thevisible range, or, outside the range sensed by the light sensing device130.

In some embodiments, the system 2100 may include the processing unit 150and the optical subsystem 105. The optical subsystem 105 is positionedalong the input optical path IOP and configured to operate on (e.g.,refract) the incident light stream before it is provided to the lightmodulation unit.

In some embodiments, the system 2100 may include a first actuatorconfigured to adjust an optical distance d_(OM) between the opticalsubsystem 105 and the light modulation unit 110 (e.g., by translatingthe light modulation unit). The processing unit may be configured tocompute an optimal value V_(OM) (value of best focus) for the opticaldistance d_(OM) based on the range (determined by the range finder) andto direct the first actuator to set the optical distance d_(OM) to theoptimal value V_(OM).

In some embodiments, the system 2100 may also include a second actuatorconfigured to adjust an optical distance d_(MS) between the lightmodulation unit and the light sensing device. The processing unit may beconfigured to compute an optimal value V_(MS) for the optical distanced_(MS) based on the optimal value V_(OM) for the optical distance d_(OM)(or based directly on the determined range) and to direct the secondactuator to set the optical distance d_(MS) to the optimal value V_(MS).

The functional relationships between any two of the range, d_(OM) andd_(MS) may be determined by calibration.

In those embodiments where the system 2100 includes the first actuator(that is configured to adjust the optical distance d_(OM) between theoptical subsystem and the light modulation unit), the processing unitmay be configured to: compute a first adjustment value ΔV₁ (e.g., anadjustment value that will translate to the optimal value from thecurrent value) for the optical distance d_(OM) based on the determinedrange, and direct the first actuator to adjust the optical distanced_(OM) according to the first adjustment value.

Furthermore, in those embodiments that also include the second actuator(configured to adjust the optical distance d_(MS) between the lightmodulation unit and the light sensing device), the processing unit maybe configured to compute a second adjustment value ΔV₂ (e.g., anadjustment value that will translate to the optimal value from thecurrent value) for the optical distance d_(MS) based on the firstadjustment value ΔV₁ (or based on the determined range) and to directthe second actuator to adjust the optical distance d_(MS) according tothe second adjustment value.

In some embodiments, the system 2100 may include a rotational actuatorconfigured to adjust an angular orientation of the light sensing device.The processing unit may be configured to compute an angle adjustmentvalue AO for the angular orientation based on the first adjustment valueΔV₁ (or based on the determined range) and to direct the rotationalactuator to adjust the angular orientation according to the angleadjustment value.

In some embodiments, the system 2100 may include an actuator A_(F)configured to adjust a focus setting of the optical subsystem 105. Theprocessing unit may be configured to: compute an optimal value V_(FS) (avalue of best focus) for the focus setting based on the determinedrange; and direct the actuator to set the focus setting to the optimalvalue V_(FS). Alternatively, the processing unit may be configured todirect the actuator A_(F) to adjust said focus setting by a certainamount Δ_(FS) that is based on the determined range and on the currentvalue of the focus setting.

In some embodiments, the system 2100 may also include an actuator A_(MS)configured to adjust the optical distance d_(MS) between the lightmodulation unit and the light sensing device (e.g., by translating thelight sensing device). The processing unit may be configured to computean optimal value V_(MS) for the optical distance d_(MS) based on theoptimal value V_(FS) for the focus setting (or based directly on thedetermined range) and to direct the actuator A_(MS) to set the opticaldistance d_(MS) to the optimal value V_(MS).

In some embodiments, the ranger finder 2110 transmits a modulated streamof monochromatic light, e.g., laser light.

As various described above, the control unit 120 is configured to drivethe array of mirrors through a sequence of spatial patterns, e.g., aspart of the compressive acquisition of images. The range finder may beconfigured to transmit the light signal and receive the reflected lightsignal while the array of mirrors is being driven through the sequenceof spatial patterns, i.e., not requiring all the mirrors to rest in thesecond orientation state (the off state). (However, in some embodimentsor in some modes of operation, all the mirrors may indeed be put intothe second orientation state to facilitate determination of range.)Thus, the acquisition of range information may operate in parallel withthe compressive acquisition of images.

In some embodiments, the sequence of samples (generated by the lightsensing device 130) corresponds to M of the spatial patterns, and thesequence of samples is usable to construct an N-pixel image, where M isless than N. Thus, the system may operate compressively, e.g., asvariously described above.

In some embodiments, the range finder 2110 is configured to determine aDoppler shift between the transmitted light signal and the reflectedlight signal. The processing unit is configured to compute a rangevelocity based on the Doppler shift. The processing unit may then directone or more actuators of the CI device to make continuous changes focusparameters such as the focus setting of optical subsystem 105, theoptical distance d_(OM) and the optical distance d_(MS).

Dual TIR Prism to Separate Incident Light from On-State Reflection andOff-State Reflection

In one set of embodiments, a system 2200 may be configured as shown inFIG. 22. System 2200 includes the light modulation unit 110M asdescribed above, and may also include a dual TIR prism 2210, a firstlight sensing device 130A and a second light sensing device 130B.(Furthermore, system 2200 may also include any subset of the features,embodiments and elements described herein.)

The light modulation unit 110M includes an array of mirrors, where eachof the mirrors is configured to controllably switch between twoorientation states, e.g., as variously described above.

The dual TIR prism 2210 has a front surface SF, a back surface SB, afirst partially-internal surface K1, a second partially-internal surfaceK1, a first exiting surface E1 and a second exiting surface E2.

The dual TIR prism 2210 is configured to receive an incident light beamL at the front surface SF and output the incident light beam at the backsurface SB. The dual TIR prism is further configured to receive a firstmodulated light beam MLB₁ at the back surface SB and from the array ofmirrors, totally internally reflect the first modulated light beam atthe first partially-internal surface K1, and output the first modulatedlight beam onto a first sensing path at the first exiting surface E1.The dual TIR prism is further configured to receive a second modulatedlight beam MLB₂ at the back surface SB and from the array of mirrors,totally internally reflect the second modulated light beam at the secondpartially-internal surface K2, and output the second modulated lightbeam onto a second sensing path at the second exiting surface E2. Thefirst modulated light beam MLB₁ comprises pieces of the incident lightbeam L that are reflected by mirrors in a first of the two orientationstates. The second modulated light beam MLB₂ comprises pieces of theincident light beam that are reflected by mirrors in a second of the twoorientation states.

In some embodiments, the first modulated light beam MLB₁ and the secondmodulated light beam MLB₂ may have angles of incidence upon the backsurface that are equal in magnitude but opposite in sign. In otherembodiments—for example, embodiments where the DMD is tilted at an anglewith respect to the imaging path axis, MLB₁ and MLB₂ may have angles ofincidence that are different in magnitude.

For more information on dual TIR prisms, please see U.S. patentapplication Ser. No. 12/665,237 (Publication No. 2010/0189344 A1), PCTfiled on Jun. 18, 2008, invented by Dirk L. A. Maes. See also “Dual TIRprism, a way to boost the performance of a DLP projector”, SID MECSpring Meeting 2008, 13 Mar. 2008, Dirk Maes, available athttp://www.ioffraunhofer.de/sid/_media/dirk_maes.pdf.

The first light sensing device 130A may be configured to receive atleast a portion of the first modulated light beam MLB₁ from the firstsensing path, and generate a first electrical signal representingintensity of the “at least a portion” of the first modulated light beam.The first light sensing device 130A may an instance of the light sensingdevice 130 described above, in any of its various embodiments.

The second light sensing 130B may be device configured to receive atleast a portion of the second modulated light beam MLB₂ from the secondsensing path, and generate a second electrical signal representingintensity of the “at least a portion” of the second modulated lightbeam. The first light sensing device 130B may an instance of the lightsensing device 130 described above, in any of its various embodiments.

In some embodiments, the front surface SF is substantially perpendicularto a central axis of the incident light beam, or, to a central axis ofthe optical subsystem 105.

In some embodiments, the back surface SB is substantially parallel to aplane of the array of mirrors.

In some embodiments, the front and back surfaces are parallel, where thefirst and second partially-internal surfaces have mirror-image symmetrywith respect to a plane orthogonal to the first and second surfaces.

In some embodiments, system 2200 includes the optical subsystem 105(e.g., a camera lens) as described above, e.g., as shown in FIG. 23.

The input optical subsystem 105 may be configured to receive theincident light beam L and to provide the incident light beam to thefront surface of the dual TIR prism, e.g., as shown in FIG. 23.

The input optical subsystem may be configured so that an image carriedby the incident light beam is in focus at the array of mirrors. In someembodiments, the input optical subsystem has an adjustable focusmechanism so that it can focus on objects at a range of distances fromthe system.

Off-State Image Sensor to Collect Images for Focusing On-State Detector

In one set of embodiments, a system 2400 may be configured as shown inFIG. 24. The system 2400 may include the light modulation unit 110M andthe light sensing device as described above, and may also include abandpass filter 2410, a bandpass filter 2420 and an image sensor 2430.

The light modulation unit 110M is configured to receive an incidentlight stream L from an optical input path IOP, e.g., as variouslydescribed above. The light modulation unit includes an array of mirrorsconfigured to receive respective portions of the incident light stream.Each of the mirrors is configured to (a) reflect the respective portionof the incident light stream onto a first optical path P₁ when themirror is in a first orientation state and (b) reflect the respectiveportion of the incident light stream onto a second optical path P₂ whenthe mirror is in a second orientation state.

The bandpass filter 2410 is configured to restrict a first light streamS₁ to a first wavelength range in order to produce a restricted lightstream R₁. The first light stream S₁ comprises the portions of theincident light stream that are reflected onto the first optical path P₁by mirrors in the first orientation state. Bandpass filters are wellknown in the field of optics and need not be explained here. In otherembodiments, the first light stream may be restricted to the firstwavelength range by means other than a bandpass filter.

The light sensing device 130 is configured to receive the restrictedlight stream R₁ and generate a sequence of samples representingintensity of the restricted light stream as a function of time.

The bandpass filter 2420 is configured to restrict a second light streamS₂ to a visible wavelength range (i.e., a wavelength range with thevisible wavelength band) in order to obtain a visible light stream V.The second light stream S₂ comprises the portions of the incident lightstream that are reflected onto the second optical path P₂ by the mirrorsin the second orientation state.

The image sensor 2430 is configured to acquire a sequence of images ofthe visible light stream V. The image sensor includes an array of lightdetecting elements. The array is preferable a two-dimensional array.However, in alternative embodiments, the array may be a one-dimensionalarray.

In some embodiments, the image sensor 2430 is a CCD array. In otherembodiments, the image sensor is a CMOS image sensor. In yet otherembodiments, the image sensor is based on some other semiconductortechnology.

In some embodiments, the first wavelength range and the visiblewavelength range are non-overlapping.

In some embodiments, the first wavelength range and the visiblewavelength range are overlapping but non-identical.

In some embodiments, the first bandpass filter is configured so that thefirst wavelength range is adjustable (or selectable), e.g.,programmable.

In some embodiments, the light modulation unit and elements downstreamfrom the light modulation unit (e.g., the light sensing device 130, theimage sensor 2430, any optics intervening between the light modulationunit 110M and the light sensing device, and any optics interveningbetween the light modulation unit 110M and the image sensor) may bemounted on a first translation stage that is translatable with respectto the optical subsystem 105. Furthermore, in some embodiments, theimage sensor 2430 is mounted on a second translation stage which isitself mounted on the first translation stage, and the light sensingdevice 130 is mounted on a third translation stage which is itselfmounted on the first translation stage. The second translation stage iselectrically and/or manually adjustable to translate the image sensor,in order to change the optical distance between the light modulationunit and the image sensor. The third translation stage is electricallyand/or manually adjustable to translate the light sensing device, inorder to change the optical distance between the light modulation unitand the light sensing device.

Change Optical Distance to Light Sensing Device in Response to in Changein First Wavelength Range

In some embodiments, system 2400 may also include an actuator configuredto adjust an optical distance between the light modulation unit and thelight sensing device. The processing unit 150 may be configured to:receive user input specifying a selection (e.g., a selection differentfrom the current selection) for the first wavelength range; determine anadjustment value for the optical distance based on the specifiedselection; and direct the actuator to adjust the optical distance basedon the determined adjustment value. The functional relationship betweenoptimal values of the optical distance and wavelength (or wavelengthrange) will have been determined by calibration and stored in a memoryaccessible to the processing unit. The functional relationship may takethe form of a lookup table, or a set of coefficients for substitutioninto a known functional form.

In some embodiments, system 2400 may also include the optical subsystem105 and the processing unit 150 as described above, and may also includea display (or an interface for coupling to a display). The opticalsubsystem 105 may be configured to receive and operate on the incidentlight stream prior to the incident light stream arriving at the lightmodulation unit. The processing unit 150 may be configured to displaythe sequence of images on the display (e.g., in real time).

User Looks at Display and Provides Input that Determines FocusAdjustment

In some embodiments, the system 2400 may also include a first actuatorconfigured to adjust a focus setting of the optical subsystem 105. Theprocessing unit 150 may be configured to: receive user input specifyinga focus adjustment; direct the first actuator to adjust the focussetting of the optical subsystem 105 based on the user-specified focusadjustment.

In some embodiments, system 2400 may also include an actuator configuredto adjust a pre-modulation optical distance d_(OM) between the opticalsubsystem 105 and the light modulation unit 110M (e.g., by translatingthe light modulation unit). The processing unit may be configured to:receive user input specifying a focus adjustment; and direct theactuator to adjust the pre-modulation optical distance d_(OM) based onthe user-specified focus adjustment.

Changing Post-Modulation Optical Distance(s) in Response to Change inPre-Modulation Optical Distance

In some embodiments, the processing unit 150 may be further configuredto: compute a first adjustment value for a first post-modulation opticaldistance d_(MI) between the light modulation unit and the image sensorbased on the user-specified focus adjustment; compute a secondadjustment value for a second post-modulation optical distance d_(MS)between the light modulation unit 110M and the light sensing device 130based on the first adjustment value (or based directly on theuser-specified focus adjustment); direct a first additional actuator toadjust the first post-modulation optical distance d_(MI) based on thefirst adjustment value; and direct a second additional actuator toadjust the second post-modulation optical distance d_(MS) based on thesecond adjustment value.

In some embodiments, system 2400 may include a first actuator configuredto adjust a first optical distance between the light modulation unit andthe image sensor (e.g., by translating image sensor), and a secondactuator configured to adjust a second optical distance between thelight modulation unit and the light sensing device. In this context, theprocessing unit 150 may be configured to: receive user input specifyinga focus adjustment for the first optical distance; direct the firstactuator to adjust the first optical distance based on theuser-specified focus adjustment; determine an adjustment value for thesecond optical distance based on the user-specified focus adjustment;and direct the second actuator to adjust the second optical distancebased on the determined adjustment value. The functional relationshipbetween the optimal value of the first optical distance and the optimalvalue for the second optical distance will have been previouslydetermined by calibration and stored in memory accessible to theprocessing unit 150.

In some embodiments, system 2400 may also include the control unit 150described above. The control unit 150 is configured to drive the arrayof mirrors through a sequence of spatial patterns. The sequence ofspatial patterns may include a first subsequence, where each spatialpattern of the first subsequence specifies that all the mirrors be inthe second orientation state (the OFF state), to allow the image sensorto capture an image (or images) of the incident light stream withoutspatial modulation. The user will then be able to see on the display adepiction of the external environment. In some embodiments, the sequenceof spatial patterns may also include a second subsequence of measurementpatterns. The samples captured by the light sensing device 130 inresponse to the assertion of the measurement patterns may be used toalgorithmically construct images corresponding to the first wavelengthrange. The image sensor may be configured to acquire imagescorresponding to the patterns of the first subsequence and to ignore ordiscard images corresponding to the patterns of the second subsequence.

In some embodiments, system 2400 may also include the optical subsystem105 and the processing unit 150. The optical subsystem may be configuredto receive and operate on the incident light stream prior to theincident light stream arriving at the light modulation unit. Theprocessing unit may be configured to compute a focus indicator valuebased on at least one of the images from the image sensor. The focusindicator value indicates an extent to which the visible light stream isin focus at the image sensor. The focus indicator value may be computedby evaluating one or more focus metrics on the image, e.g., as variouslydescribed above. The one or more focus metrics may include one or more2D focus metrics and/or one or more 1D focus metrics. A 2D focus metricis a focus metric that operates on a 2D image. A 1D focus metric is afocus metric that operates on a 1D image. (Even though the image fromthe image sensor may be a 2D image, it is possible to derive a 1D imagefrom the 2D image by extracting pixel values along a 1D path or a set of1D paths in the 2D image.)

In some embodiments, system 2400 may also include a first actuatorconfigured to adjust a focus setting of the optical subsystem 105. Inthis context, the processing unit may be configured to direct the firstactuator to adjust the focus setting of the optical subsystem based ondata including the focus indicator value. The data may also includes apreviously-determined focus indicator value for a previous state of thefocus setting and the value of that previous focus setting as well asthe value of the current focus setting.

In some embodiments, system 2400 may also include a first actuatorconfigured to adjust a first optical distance between the opticalsubsystem 105 and the light modulation unit 110M (e.g., by translatingthe light modulation unit or by translating the optical subsystem). Inthis context, the processing unit 150 may be configured to: compute afirst adjustment value for the first optical distance based on dataincluding the focus indicator value; and direct the first actuator toadjust the first optical distance based on the first adjustment value.

In some embodiments, the processing unit 150 may be further configuredto: compute a second adjustment value for a second optical distancebetween the light modulation unit and the image sensor based on thefirst adjustment value; compute a third adjustment value for a thirdoptical distance between the light modulation unit and the light sensingdevice based on the first adjustment value (e.g., using a knownfunctional relationship between the optimal value of second opticaldistance and the optimal value of the third optical distance, havingbeen determined by calibration); direct a second actuator to adjust thesecond optical distance based on the second adjustment value; and directa third actuator to adjust the third optical distance based on the thirdadjustment value.

In some embodiments, system 2400 may also include a first actuatorconfigured to adjust a first optical distance between the lightmodulation unit and the image sensor (e.g., by translating the imagesensor) and a second actuator configured to adjust a second opticaldistance between the light modulation unit and the light sensing device.In this context, the processing unit may be configured to: compute afirst adjustment value for the first optical distance based on dataincluding the focus indicator value; compute a second adjustment valuefor the second optical distance based on the first adjustment value(using a known functional relationship between the optimal value of thefirst optical distance and the optimal value of the second opticaldistance); direct the first actuator to adjust the first opticaldistance based on the first adjustment value; and direct the secondactuator to adjust the second optical distance based on the secondadjustment value.

In some embodiments, system 2400 may also include the optical subsystem105, the control unit 120 and the processing unit 150, and may alsoinclude a first actuator. The first actuator may be configured to adjusta focus setting of the optical subsystem. The control unit may beconfigured to drive the array of mirrors through a sequence of spatialpatterns. The processing unit may be configured to operate on a firstsubset of the sequence of images acquired by the image sensor todetermine focus indicator values, where each of focus indicator valuesis determined based on one or more of the images of the first subset,where each of the focus indicator values indicates an extent to whichthe visible light stream is in focus at the image sensor. The processingunit may be configured to drive the first actuator based on the focusindicator values in order to locate an improved value (e.g., optimalvalue) of the focus setting.

In some embodiments, the spatial patterns include one-dimensionalpatterns, where each of the one-dimensional patterns is restricted to acorresponding one-dimensional path on the array of the mirrors.

In some embodiments, system 2400 may also include the control unit 120and the processor 150, and may also include a first actuator and asecond actuator. The first actuator may be configured to adjust a firstoptical path length between the light modulation unit and the imagesensor. The second actuator may be configured to adjust a second opticalpath length between the light modulation unit and the light sensingdevice. The control unit may be configured to drive the array of mirrorsthrough a sequence of spatial patterns, where the sequence of spatialpatterns includes a plurality of repetitions of a collection of one ormore calibration patterns. The one or more calibration patternscorrespond respectively to one or more subsets of the array of mirrors.Each of the one or more subsets corresponds to a respective subset ofthe array of light detecting elements. The sequence of images acquiredby the image sensor includes an image for each of the spatial patterns.

The processing unit may be configured to compute a sequence of spillovervalues, where each of the spillover values corresponds to respective oneof the plurality of repetitions of the set of one or more calibrationpatterns and is computed based on one or more of the images thatcorrespond to the respective repetition. Each spillover value indicatesan extent to which modulated light from the one or more subsets of thearray of mirrors fail to reach their respective subsets of the array oflight detecting elements.

In some embodiments, the processing unit 150 may be configured to: drivethe first actuator based on the spillover values in order to determinean improved value (or optimal value) for the first optical path length;compute an improved value for the second optical path length based onthe improved value for the first optical path length; direct the secondactuator to adjust the second optical path length based on the improvedvalue for the second optical path length.

In some embodiments, system 2400 may be configured as shown in FIG. 25.Mirror 117M is configured to reflect the first light stream S₁ onto apath leading to the bandpass filter 2410. Likewise, mirror 118M isconfigured to reflect the second light stream S₂ onto a path leading tothe bandpass filter 2420. Furthermore, system 2400 may include opticsintervening between the light modulation unit and the image sensor todirect the visible light stream V onto the image sensor, and opticsintervening between the light modulation unit and the light sensingdevice 130 to direct the restricted light stream R₁ onto the lightsensing device.

In some embodiments, system 2400 may be configured as shown in FIG. 26.A TIR prism pair 2610 is employed to separate the incident light streamL from the first light stream S₁ and the second light stream S₂. The TIRprism pair 2610 may be configured as variously described above.

In one set of embodiments, a system 2700 may be configured as shown inFIG. 27A. System 2700 includes the light modulation 110 and the lightsensing device as described above, and also includes a first opticalsubsystem 2710 and an image sensor 2720.

The optical subsystem 2710 may be configured to receive an incidentlight stream and to split (or separate) the incident light stream into afirst light stream S₁ and a second light stream S₂. Any of various meansof splitting are contemplated.

The light modulation unit 110 is configured to modulate the first lightstream with a sequence of spatial patterns to obtain a modulated lightstream MLS, e.g., as variously described above.

The light sensing device 130 is configured to receive the modulatedlight stream and to generate a sequence of samples {I(k)} representingintensity of the modulated light stream as a function of time, e.g., asvariously described above.

The image sensor 2720 may be configured to receive the second lightstream S₂ and acquire a sequence of images of the second light streamS₂. The image sensor includes an array of light detecting elements. Theimage sensor may be a low cost image sensor. In some embodiments, theimage sensor is a CCD array or CMOS image sensor.

In some embodiments, the first light stream S₁ is restricted to a firstwavelength range and the second light stream S₂ is restricted to asecond wavelength range that are different (e.g., non-overlapping, or,overlapping but non-identical). In one embodiment, the second wavelengthrange is a visible wavelength range, i.e., wavelength range in thevisible light spectrum.

In some embodiments, the system 2700 may also include the processingunit 150 as described above. The processing unit may be configured tooperate on the samples {I(k)} and/or to operate on the images orselected ones of the images.

In some embodiments, the system 2700 may include a bandpass filterconfigured to receive and the modulated light stream and to restrict themodulated light stream to a first wavelength range. The light sensingdevice 130 receives the modulated light stream as restricted to thefirst wavelength range by the bandpass filter.

In some embodiments, the bandpass filter is configured so that the firstwavelength range is adjustable (or selectable).

In some embodiments, system 2700 may also include an actuator configuredto adjust an optical distance between the light modulation unit and thelight sensing device 130, e.g., by translating the light sensing devicealong a central axis of the modulated light stream. The processing unit150 may be configured to receive user input specifying a selection forthe first wavelength range; determine a value of best focus for theoptical distance based on the specified selection for the firstwavelength range; and direct the actuator to adjust the optical distancebased on the determined value.

The value of best focus may be determined by evaluating a calibrationfunction, e.g., a function of wavelength or wavelength range index thathas been determined by previous calibration. The function may beevaluated by evaluating a mathematical expression whose parameters arestored in system 2700, or, by table lookup.

In some embodiments, system 2700 may also include a second bandpassfilter configured to restrict the second light stream S₂ to the visiblewavelength range. The image sensor 2710 may receive the second lightstream S₂ as restricted to the visible wavelength range. Thus, theimages produced by the image sensor may be images corresponding to thevisible wavelength range.

In some embodiments, system 2700 also includes the optical subsystem105, the processing unit 150 and a display 2730, e.g., as shown in FIG.27B. The optical subsystem 105 (e.g., a camera lens) may be configuredto receive and operate on the incident light stream prior to theincident light stream arriving at the optical subsystem 2710. Theprocessing unit may be configured to display the sequence of images onthe display (e.g., in real time).

User Looks at Display and Provides Input Determining Focus Adjustment

In some embodiments, the processing unit 150 may be configured toreceive user input specifying a focus adjustment. The viewing the imageon the display, the user can judge when optimal focus has been achievedon the image sensor 2720. The processing unit may adjust a focus settingof the optical subsystem 105 (e.g., camera lens) based on theuser-specified focus adjustment.

In some embodiments, the processing unit 150 may be configured toevaluate a calibration function f_(IS2M) based on a current value of thefocus setting to obtain a modulator-specific value of the focus setting,e.g., in response to user input indicating that the image as seen on thedisplay 2730 is in focus. The calibration function f_(IS2M) relatesoptimal values of the focus setting for focusing onto the image sensorto optimal values of the focus setting for focusing onto the lightmodulation unit 110. The processing unit may then change the focussetting of the optical subsystem 105 to the modulator-specific value.

In some embodiments, the processing unit 150 may be further configuredto evaluate a calibration function f_(M2D) based on themodulator-specific value of the focus setting to obtain an optimal valuefor an optical distance between the light modulation unit 110 and thelight sensing device 130. The processing unit may then change theoptical distance to the optimal value (e.g., by translating the lightsensing device).

As noted above, the processing unit 150 may be configured to receiveuser input specifying a focus adjustment. In some embodiments, theprocessing unit 150 may change an optical distance d_(TIS) between theoptical subsystem 2710 and the image sensor 2720 based on theuser-specified focus adjustment.

In some embodiments, the processing unit 150 may be further configuredto evaluate a calibration function g_(IS2M) based on a current value ofthe optical distance d_(TIS) to obtain a value V for an optical distanced_(TM) between the optical subsystem 2710 and the light modulation unit110, e.g., in response to user input indicating that the image as seenon the display 2730 is in focus. The calibration function g_(IS2M)relates optimal values of the optical distance d_(TIS) for focusing ontothe image sensor to optimal values of the optical distance d_(TM) forfocusing onto the light modulation unit. The processing unit may thenchange the optical distance d_(TM) to the value V.

In some embodiments, the processing unit 150 may be configured toevaluate a calibration function g_(M2D) based on the value V of thesecond optical distance d_(TM) to obtain an optimal value for an opticaldistance d_(MD) between the light modulation unit 110 and the lightsensing device 130. The processing unit may then change the opticaldistance d_(MD) to the optimal value (e.g., by translating the lightsensing device).

In some embodiments, the processing unit 150 may be configured tocompute focus indicator values for respective ones of the images. Forexample, the processing unit may evaluate a 2D focus metric (or acombination of 2D focus metrics) on each image or selected ones of theimages. (In one embodiment, the processing unit may evaluate a 1D focusmetric (or a combination of 1D focus metrics) on the 1D slices throughthe images. Each focus indicator value indicates an extent to which thesecond light stream S₂ is in focus at the image sensor 2720.

In some embodiments, the processing unit 150 may be configured to changea focus setting of the optical subsystem 105 one or more times until thefocus indicator value is optimized (e.g., maximized or minimized). Theoptimizing state of the focus setting may be the state where optimalfocus has been achieved for the light on the image sensor 2720.

In some embodiments, the processing unit 150 may be configured toevaluate the calibration function f_(IS2M) based on a first value of thefocus setting to obtain a second value of the focus setting (e.g., inresponse to having optimized the focus indicator value on the imagesfrom the image sensor). As described above, the calibration functionf_(IS2M) relates optimal values of the focus setting for focusing ontothe image sensor to optimal values of the focus setting for focusingonto the light modulation unit. The processing unit may then change thefocus setting of the optical subsystem 105 to the second value.

In some embodiments, the processing unit 150 may be configured toevaluate the calibration function f_(M2D) based on the second value ofthe focus setting to obtain an optimal value for the optical distanced_(MD) between the light modulation unit and the light sensing device.The processing unit may then change the optical distance d_(MD) to theoptimal value (e.g., by translating the light sensing device).

In some embodiments, the processing unit 150 may be configured to changethe first optical distance d_(TIS) between the optical subsystem 2710and the image sensor one or more times until the focus indicator valueon the image-sensor images is optimized (e.g., maximized or minimized).The optimizing value of the distance d_(TIS) may be the value whereoptimal focus has been achieved for the light on the image sensor 2720

In some embodiments, the processing unit 150 may be configured toevaluate the calibration function g_(IS2M) based on an optimizing valueof the optical distance d_(TIS) to obtain an optimal value V for theoptical distance d_(TM) between the optical subsystem 2710 and the lightmodulation unit 110. As noted above the calibration function g_(IS2M)relates optimal values of the optical distance d_(TIS) for focusing ontothe image sensor to optimal values of the optical distance d_(TM) forfocusing onto the light modulation unit. The processing unit may thenchange the optical distance d_(TM) to the second value V.

In some embodiments, the processing unit 150 may be configured toevaluate the calibration function g_(M2D) based on the optimal value Vof the optical distance d_(TM) to obtain an optimal value V_(MD) for theoptical distance d_(MD) between the light modulation unit and the lightsensing device. The processing unit may then change the optical distanced_(MD) to the optimal value V_(MD) (e.g., by translating the lightsensing device).

Pre-Modulation Separation of Light for Image Sensor

In some embodiments of system 2700, the light modulation unit comprisesan array of mirrors 110M, e.g., as shown in FIG. 27C. The mirrors areconfigured to receive respective portions of the first light stream S₁.Each of the mirrors is configured to (a) reflect the respective portionof the first light stream S₁ onto a first optical path P₁ when themirror is in a first orientation state and (b) reflect the respectiveportion of the first light stream away from the first optical path whenthe mirror is in a second orientation state.

The light sensing device 130 is configured to receive a modulated lightstream from the first optical path P₁. The modulated light streamcomprises the portions of the first light stream that are reflected ontothe first optical path by mirrors in the first orientation state. Thefirst light sensing device is further configured to generate a sequenceof samples {I(k)} representing intensity of the modulated light streamas a function of time.

In some embodiments, the optical subsystem 2710 includes a TIR prismpair. A first prism of the TIR prism pair may be configured to generatethe first light stream S₁ by partially transmitting the incident lightstream and generate the second light stream S₂ by partially reflectingthe incident light stream.

In some embodiments, a first surface of the first prism is coated with adielectric coating in order to restrict the second light stream S₂ to avisible wavelength range.

In some embodiments, each of the mirrors (of the light modulating array)is configured to reflect the respective portion of the first lightstream S₁ onto a second optical path P₂ when the mirror is in the secondorientation state. In these embodiments, system 2700 may also include asecond light sensing device configured to receive a complementarymodulated light stream from the second optical path P₂. Thecomplementary modulated light stream comprises the portions of the firstlight stream S₁ that are reflected onto the second optical path bymirrors in the second orientation state. The second light sensing devicemay be further configured to generate a second sequence of samplesrepresenting intensity of the complementary modulated light stream as afunction of time. For further information regarding compressive imagingsystems that use both a modulated light stream and a complementarymodulated light stream reflected from a light modulation unit, see U.S.patent application Ser. No. 13/207,900, filed on Aug. 11, 2011.

FIG. 28D shows an embodiment where light is split at the interface ofthe first prism of the TIR prism pair 2710. In this embodiment, adielectric coating on this interface reflects visible light andtransmits the remaining spectral components of the light. The reflectedvisible light forms an image on the sensor array 2720. The positions ofthe sensor array and the DMD 110M are calibrated such that when theimage is in focus on the sensor array at visible wavelengths it is alsoin focus on the DMD at the remaining wavelengths.

Compressive Imaging System 2800

In one set of embodiments, a compressive imaging system 2800 may beconfigured as shown in FIG. 28. The compressive imaging (CI) system mayinclude an optical system 2810, a spatial light modulator 2815, a set2820 of one or more photodetectors, a set 2825 of one or more amplifiers(i.e., one amplifier per detector), a set 2830 of analog-to-digitalconverters (one ADC per detector), and a processing element 2840.

The optical system 2810 focuses an incident light stream onto thespatial light modulator, e.g., as variously described above. See thediscussion above regarding optical subsystem 105. The incident lightstream carries an image (or a spectral ensemble of images) that is to becaptured by the CI system in compressed form.

The spatial light modulator 2815 modulates the incident light streamwith a sequence of spatial patterns to obtain a modulated light stream,e.g., as variously described above.

Each of the detectors 2820 generates a corresponding electrical signalthat represents the intensity of a corresponding portion of themodulated light stream, e.g., a spatial portion or a spectral portion ofthe modulated light stream.

Each of the amplifiers 2825 (e.g., transimpedance amplifiers) amplifiesthe corresponding detector signal to produce a corresponding amplifiedsignal.

Each of the ADCs 2830 acquires samples of the corresponding amplifiedsignal.

The processing element 2840 may operate on the sample sets obtained bythe respective ADCs to construct respective images. The images mayrepresent spatial portions or spectral slices of the incident lightstream. Alternatively, or additionally, the processing element may sendthe sample sets to a remote system for image construction.

The processing element 2840 may include one or more microprocessorsconfigured to execute program instructions stored in a memory medium.

The processing element 2840 may be configured to control one or moreother elements of the CI system. For example, in one embodiment, theprocessing element may be configured to control the spatial lightmodulator, the transimpedance amplifiers and the ADCs.

The processing element 2840 may be configured to perform any subset ofthe above-described methods on any or all of the detector channels. Eachof the detector channels of system 2800 may include any subset of theembodiments, features, and elements described above. (A detector channelmay include a corresponding detector, amplifier and ADC.)

Compressive Imaging System 2900

In one set of embodiments, a compressive imaging system 2900 may beconfigured as shown in FIG. 29. The compressive imaging system includesthe light modulation unit 110 as variously described above, and alsoincludes optical subsystem 2910, a set of L light sensing devices LSD₁through LSD_(L), and a set of L signal acquisition channels C₁ throughC_(L), where L in a positive integer.

The light modulation unit 110 receives an incident light stream andmodulates the incident light stream with a sequence of spatial patternsto obtain a modulated light stream MLS, e.g., as variously describedabove.

The optical subsystem 2910 delivers portions (e.g., spatial portions orspectral portions) of the modulated light stream to corresponding onesof the light sensing devices LSD₁ through LDS_(L).

For information on various mechanisms for delivering spatial subsets ofthe modulated light stream to respective light sensing devices, pleasesee U.S. patent application Ser. No. 13/197,304, filed on Aug. 3, 2011,titled “Decreasing Image Acquisition Time for Compressive ImagingDevices”, invented by Woods et al., which is hereby incorporated byreference in its entirety as though fully and completely set forthherein.

In some embodiments, the optical subsystem 2910 includes one or morelenses and/or one or more mirrors arranged so as to deliver spatialportions of the modulated light stream onto respective ones of the lightsensing devices. For example, in one embodiment, the optical subsystem2910 includes a lens whose object plane is the plane of the array oflight modulating elements and whose image plane is a plane in which thelight sensing devices are arranged. The light sensing devices may bearranged in an array.

In some embodiments, optical subsystem 2910 is configured to separatethe modulated light stream into spectral components and deliver thespectral components onto respective ones of the light sensing devices.For example, optical subsystem 2910 may include a grating, aspectrometer, or a tunable filter such as a Fabry-Perot Interferometerto achieve the spectral separation.

Each light sensing device LSD_(j) generates a corresponding electricalsignal v_(j)(t) that represents intensity of the corresponding portionMLS_(j) of the modulated light stream.

Each signal acquisition channel C_(j) acquires a corresponding sequenceof samples {V_(j)(k)} of the corresponding electrical signal v_(j)(t).Each signal acquisition channel may include a corresponding amplifier(e.g., a TIA) and a corresponding A/D converter.

The sample sequence {V_(j)(k)} obtained by each signal acquisitionchannel C_(j) may be used to construct a corresponding sub-image whichrepresents a spatial portion or a spectral slice of the incident lightstream. The number of samples m in each sample sequence {V_(j)(k)} maybe less (typically much less than) the number of pixels in thecorresponding sub-image. Thus, each signal acquisition channel C_(j) mayoperate as a compressive sensing camera for a spatial portion orspectral portion of the incident light.

Each of the signal acquisition channels may include any subset of theembodiments, features, and elements described above.

System 2900 may also include the optical subsystem 2010 (or the opticalsubsystem 105) for focusing the incident light stream onto the lightmodulation unit 110.

The following numbered paragraphs describe various additionalembodiments of systems and methods.

1.1 A method for focusing a compressive imaging (CI) device, the methodcomprising:

(a) supplying a sequence of calibration patterns to a light modulationunit of the CI device, wherein the light modulation unit includes anarray of light modulating elements;

(b) the light modulation unit modulating an incident light stream withthe sequence of calibration patterns to produce a modulated lightstream, wherein the sequence of calibrations patterns is configured toeffect a movement of a region along a one-dimensional path on the arrayof light modulating elements, wherein each of the calibration patternscorresponds a different position of the region along the one-dimensionalpath;

(c) a light sensing device of the CI device receiving the modulatedlight stream and acquiring a sequence of samples representing anintensity of the modulated light stream as a function of time, whereinthe sequence of samples includes at least one sample for each of thecalibration patterns;

(d) computing a focus indicator value based on the sequence of samples,wherein the focus indicator value indicates an extent to which theincident light stream is in focus along the one-dimensional path on thearray of light modulating elements.

1.2 The method of paragraph 1.1, wherein the CI device includes anoptical subsystem that receives and operates on the incident lightstream prior to the incident light stream arriving at the lightmodulation unit.

1.3 The method of paragraph 1.2, further comprising: changing a focussetting of the optical subsystem based on data including the focusindicator value.

1.4 The method of paragraph 1.2, further comprising repeatedlyperforming a set of operations including: changing a focus setting ofthe optical subsystem; and performing actions (a), (b), (c) and (d),wherein the set of operations is repeated until the focus indicatorvalue becomes less than a threshold value.

1.5 The method of paragraph 1.2, further comprising: changing an opticaldistance between the light modulation unit and the optical subsystembased on data including the focus indicator value.

1.6 The method of paragraph 1.2, further comprising repeatedlyperforming a set of operations including: changing an optical distancebetween the light modulation unit and the optical subsystem; andperforming (a), (b), (c) and (d), wherein the set of operations isrepeated until the focus indicator value becomes less than a thresholdvalue.

1.7 The method of paragraph 1.6, further comprising: changing an opticaldistance between the light modulation unit and the light sensing devicebased on a final optical distance between the light modulation unit andthe optical subsystem after the focus indicator value becomes less thanthe threshold value.

1.8 The method of paragraph 1.1, further comprising: determining thatthe focus indicator value is inaccurate; and performing (a), (b), (c)and (d) a second time, using a second sequence of calibration patterns,to obtain a second sequence of samples and a second focus indicatorvalue, wherein the second sequence of calibration patterns is configuredto effect the movement of the region along a second one-dimensional pathon the array of light modulating elements.

1.9 The method of paragraph 1.1, wherein (a), (b), (c) and (d) areperformed a plurality of times, using a respective plurality ofcalibration sequences to obtain a respective plurality of focusindicator values for a respective plurality of one-dimensional paths onthe array of light modulating elements.

1.10 The method of paragraph 1.2, further comprising: directing anactuator to change an optical distance between the optical subsystem andthe light modulation unit according to a first displacement value;performing (a) through (d) a second time, to obtain an additional focusindicator value for an additional sequence of samples; computing asecond displacement value for the optical distance based on dataincluding the focus indicator value for the sequence of samples, thefocus indicator value for the additional sequence of samples and thefirst displacement value.

1.11 The method of paragraph 1.10, further comprising: directing theactuator to change the optical distance between the optical subsystemand the light modulation unit according to the second displacementvalue.

1.12 The method of paragraph 1.2, further comprising: directing anactuator to change a focus setting of the optical subsystem according toa first displacement value; performing (a) through (d) a second time, toobtain an additional focus indicator value for an additional sequence ofsamples; computing a second displacement value for the focus settingbased on data including the focus indicator value for the sequence ofsamples, the focus indicator value for the additional sequence ofsamples and the first displacement value.

1.13 The method of paragraph 1.12, further comprising: directing theactuator to change the focus setting of the optical subsystem accordingto the second displacement value.

1.14 The method of paragraph 1.2, further comprising: directing anactuator to change an optical distance between the optical subsystem andthe light modulation unit through a range; performing (a) through (d)for each of a plurality of optical distances within the range in orderto obtain a corresponding plurality of sample sequences and acorresponding plurality of focus indicator values; determining anoptimal value for the optical distance between the input opticalsubsystem and the light modulation unit based on an analysis of dataincluding the plurality of focus indicator values.

1.15 The method of paragraph 1.14, further comprising: directing theactuator to change the optical distance to the optimal value.

1.16 The method of paragraph 1.2, further comprising: directing anactuator to change a focus setting of the optical subsystem through arange of settings; performing (a) through (d) for each of a plurality ofsettings within the range in order to obtain a corresponding pluralityof sample sequences and a corresponding plurality of focus indicatorvalues; determining an optimal value for the focus setting based on ananalysis of data including the plurality of focus indicator values.

1.17 The method of paragraph 1.16, further comprising: directing theactuator to change the focus setting to the optimal value.

1.18 The method of paragraph 1.1, wherein the light sensing deviceincludes a plurality of light sensing elements, wherein each of thelight sensing elements generates a respective electrical signalrepresenting intensity of a respective spatial portion of the modulatedlight stream, wherein said acquiring the sequence of samples comprisesadding the electrical signals to obtain a sum signal and sampling thesum signal.

1.19 The method of paragraph 1.1, wherein the light sensing deviceincludes a plurality of light sensing elements, wherein each of thelight sensing elements generates a respective electrical signalrepresenting intensity of a respective spatial portion of the modulatedlight stream, wherein said acquiring the sequence of samples comprisessampling each of the electrical signals and adding the sampled versionsof the electrical signals in the digital domain.

1.20 The method of paragraph 1.1, wherein the light sensing deviceincludes only one light sensing element.

1.21 The method of paragraph 1.1, wherein said computing a focusindicator value includes: computing a discrete Fourier transform of aone-dimensional image derived from the sequence of samples; andcomputing an amount of energy present in the discrete Fourier transformat frequencies above a cutoff frequency.

1.22 A method for focusing a compressive imaging (CI) device, the methodcomprising:

(a) supplying a sequence of calibration patterns to a light modulationunit of the CI device, wherein the light modulation unit includes anarray of light modulating elements;

(b) the light modulation unit modulating an incident light stream withthe sequence of calibration patterns to produce a modulated lightstream, wherein the sequence of calibrations patterns is configured toeffect a movement of a region along a one-dimensional path on the lightmodulation unit, wherein each of the calibration patterns corresponds adifferent position of the region along the one-dimensional path;

(c) acquiring a sequence of the frames from an array of light sensingelements, wherein each of the light sensing elements is configured toreceive a respective spatial portion of the modulated light stream,wherein each of the frames of the acquired sequence corresponds to arespective one of the calibration patterns, wherein each of the framesincludes a sample from each of the light sensing elements,

(d) determining a sequence of sum values, wherein each of the sum valuesis determined by adding the samples in a respective one of the frames ofthe acquired sequence;

(e) operating on the sequence of sum values to determine a focusindicator value, wherein the focus indicator value indicates an extentto which the incident light stream is in focus along the one-dimensionalpath.

1.23 A method for focusing a compressive imaging (CI) device, the methodcomprising:

(a) supplying a sequence of calibration patterns to a light modulationunit of the CI device, wherein the light modulation unit includes anarray of light modulating elements;

(b) the light modulation unit modulating an incident light stream withthe sequence of calibration patterns to produce a modulated lightstream;

(c) a light sensing device of the CI device receiving the modulatedlight stream and acquiring a sequence of samples representing anintensity of the modulated light stream as a function of time, whereinthe sequence of samples includes at least one sample for each of thecalibration patterns;

(d) changing a focus setting of the CI device based on the sequence ofsamples.

1.24 The method of paragraph 1.23, wherein the focus setting of the CIdevice is an optical distance between an optical subsystem of the CIdevice and the light modulation unit.

1.25 The method of paragraph 1.23, wherein the focus setting of the CIdevice is a focus setting of an optical subsystem of the CI device.

1.26 The method of paragraph 1.23, further comprising: repeating (a)through (d) until a focus metric based on the sequence of samples isoptimized.

1.27 The method of paragraph 1.23, wherein the sequence of calibrationspatterns is configured to effect the movement of a region along aone-dimensional path on a surface of the light modulation unit, whereineach of the calibration patterns corresponds a different position of theregion along the one-dimensional path.

1.28 A method for focusing a compressive imaging (CI) device, the methodcomprising:

(a) supplying a sequence of calibration patterns to a light modulationunit of the CI device, wherein the light modulation unit includes anarray of light modulating elements;

(b) the light modulation unit modulating an incident light stream withthe sequence of calibration patterns to produce a modulated lightstream, wherein the sequence of calibrations patterns is configured toeffect a movement of a region along a one-dimensional path on the lightmodulation unit, wherein each of the calibration patterns corresponds adifferent position of the region along the one-dimensional path;

(c) a light sensing device of the CI device receiving the modulatedlight stream and acquiring a sequence of samples representing anintensity of the modulated light stream as a function of time, whereinthe sequence of samples includes at least one sample for each of thecalibration patterns;

(d) displaying a one-dimensional image on a display, wherein theone-dimensional image is based on the sequence of samples;

(e) adjusting a focus setting of the CI device based on user input.

1.29 The method of paragraph 1.28, wherein the user input specifies adirection and/or magnitude of the adjustment to the focus setting.

1.30 The method of paragraph 1.28, wherein the focus setting of the CIdevice is a focus setting of an optical subsystem of the CI device.

1.31 The method of paragraph 1.28, wherein the focus setting of the CIdevice is an optical distance between an optical subsystem of the CIdevice and the light modulation unit.

1.32 The method of paragraph 1.28, performing (a) through (e) aplurality of times.

1.33 The method of paragraph 1.28, further comprising:

receiving user input specifying a position and/or orientation of theone-dimensional path on the array of light modulating elements.

2.1 A method for focusing a compressive imaging (CI) device, the methodcomprising:

(a) supplying a sequence of spatial patterns to a light modulation unitof the CI device;

(b) the light modulation unit modulating an incident light stream withthe sequence of spatial patterns to produce a modulated light stream;

(c) a light sensing device of the CI device receiving the modulatedlight stream and acquiring a sequence of samples representing anintensity of the modulated light stream as a function of time, whereinthe sequence of samples includes at least one sample for each of thespatial patterns;

(d) constructing an image using the sequence of spatial patterns and theacquired samples;

(e) computing a focus indicator value based on the image, wherein thefocus indicator value indicates an extent to which the incident lightstream is in focus at the light modulation unit.

2.2 The method of paragraph 2.1, wherein each of the spatial patterns isrestricted to a subset of the array of light modulating elements.

2.3 The method of paragraph 2.2, wherein the subset of the array oflight modulating elements is a convex region located at the center ofthe array of light modulating elements.

2.4 The method of paragraph 2.1, wherein the CI device includes anoptical subsystem that receives and operates on the incident lightstream prior to the incident light stream arriving at the lightmodulation unit.

2.5 The method of paragraph 2.4, further comprising: changing a focussetting of the optical subsystem based on data including the focusindicator value.

2.6 The method of paragraph 2.5, further comprising repeatedlyperforming a set of operations that includes: changing a focus settingof the optical subsystem; and performing (a), (b), (c), (d) and (e),wherein the set of operations is repeated until the focus indicatorvalue becomes less than a threshold value.

2.7 The method of paragraph 2.4, further comprising: changing an opticaldistance between the optical subsystem and the light modulation unitbased on data including the focus indicator value.

2.8 The method of paragraph 2.4, further comprising repeatedlyperforming a set of operations that includes: changing an opticaldistance between the optical subsystem and the light modulation unit;and performing (a), (b), (c), (d) and (e), wherein the set of operationsis repeated until the focus indicator value becomes less than athreshold value.

2.9 The method of paragraph 2.8, further comprising: changing an opticaldistance between the light modulation unit and the light sensing devicebased on a final distance between the optical subsystem and the lightmodulation unit after the focus indicator value becomes less than thethreshold value.

2.10 The method of paragraph 2.1, further comprising: changing an angleof orientation of the light modulation unit based on data including thefocus indicator value.

2.11 The method of paragraph 2.4, further comprising: directing anactuator to change an optical distance between the optical subsystem andthe light modulation unit according to a first displacement value;performing (a) through (e) a second time, to obtain an additional imageand a focus indicator value for the additional image; computing a seconddisplacement value for the optical distance based on data including thefocus indicator value for the image, the focus indicator value for theadditional image and the first displacement value.

2.12 The method of paragraph 2.8, further comprising: directing theactuator to change the optical distance according to the seconddisplacement value.

2.13 The method of paragraph 2.4, further comprising: directing anactuator to change a focus setting of the optical subsystem according toa first displacement value; performing (a) through (e) a second time, toobtain an additional image and a focus indicator value for theadditional image; computing a second displacement value for the focussetting based on data including the focus indicator value for the image,the focus indicator value for the additional image and the firstdisplacement value.

2.14 The method of paragraph 2.13, further comprising: directing theactuator to change the focus setting of the optical subsystem accordingto the second displacement value.

2.15 The method of paragraph 2.4, further comprising: directing anactuator to change an optical distance between the optical subsystem andthe light modulation unit through a range of distances; performing (a)through (e) for each of a plurality of distances within the range inorder to obtain a corresponding plurality of images and a correspondingplurality of focus indicator values; determining an optimal value forthe optical distance based on an analysis of data including theplurality of focus indicator values.

2.16 The method of paragraph 2.15, further comprising: directing theactuator to change the optical distance between the optical subsystemand the light modulation unit to the optimal value.

2.17 The method of paragraph 2.4, further comprising: directing anactuator to change the focus setting of the optical subsystem through arange of settings; performing (a) through (e) for each of a plurality offocus settings within the range in order to obtain a correspondingplurality of images and a corresponding plurality of focus indicatorvalues; determining an optimal value for the focus setting based on ananalysis of data including the plurality of focus indicator values.

2.18 The method of paragraph 2.17, further comprising: directing theactuator to change the focus setting to the optimal value.

2.19 The method of paragraph 2.1, wherein the light sensing deviceincludes a plurality of light sensing elements, wherein each of thelight sensing elements generates a respective electrical signalrepresenting intensity of a respective spatial portion of the modulatedlight stream, wherein said acquiring the sequence of samples comprisesadding the electrical signals to obtain a sum signal and sampling thesum signal.

2.20 The method of paragraph 2.1, wherein the light sensing deviceincludes a plurality of light sensing elements, wherein each of thelight sensing elements generates a respective electrical signalrepresenting intensity of a respective spatial portion of the modulatedlight stream, wherein said acquiring the sequence of samples comprisessampling each of the electrical signals and adding the sampled versionsof the electrical signals in the digital domain.

2.21 The method of paragraph 2.1, wherein the light sensing deviceincludes only one light sensing element.

2.22 A method for focusing a compressive imaging (CI) device, the methodcomprising:

(a) supplying a sequence of spatial patterns to a light modulation unitof the CI device;

(b) the light modulation unit modulating an incident light stream withthe sequence of spatial patterns to produce a modulated light stream;

(c) acquiring a sequence of the frames from an array of light sensingelements of the CI device, wherein each of the light sensing elementsreceives a respective spatial portion of the modulated light stream,wherein each of the frames corresponds to respective one of the spatialpatterns, wherein each of the frames includes one sample from each ofthe light sensing elements;

(d) determining a sequence of sum values, wherein each of the sum valuesis determined by adding the samples in a respective one of the frames ofthe acquired sequence;

(e) constructing an image using the sequence of spatial patterns and thesequence of sum values;

(f) computing a focus indicator value based on the image, wherein thefocus indicator value indicates an extent to which the incident lightstream is in focus at the light modulation unit.

2.23 A method for determining focus-related information for acompressive imaging (CI) device, the method comprising:

(a) modulating an incident light stream with the sequence of spatialpatterns, thereby generating a modulated light stream, wherein saidmodulating is performed by a light modulation unit of the CI device;

(b) generating an electrical signal representing intensity of themodulated light stream as a function of time, wherein the electricalsignal is generated by a light sensing device of the CI device;

(c) acquiring samples of the electrical signal;

(d) constructing an image using the sequence of spatial patterns and asubset of the acquired samples that corresponds to the sequence ofspatial patterns;

(e) computing focus information based on the image, wherein the focusinformation comprises information regarding at extent to which theincident light stream is in focus at the light modulation unit.

2.24 The method of paragraph 2.23, wherein the number of spatialpatterns in the sequence of spatial patterns is m, wherein the image isan n-pixel image with n less than the number N of light modulatingelements in the light modulation unit, wherein m is less than n.

2.25 The method of paragraph 2.23, further comprising: changing anoptical distance between an optical subsystem of the CI device and thelight modulation unit based on the focus information.

2.26 The method of paragraph 2.23, further comprising: changing a focussetting of an optical subsystem of the CI device based on the focusinformation.

2.27 A method for focusing a compressive imaging (CI) device, the methodcomprising:

(a) modulating an incident light stream with a sequence of spatialpatterns to obtain a modulated light stream, wherein said modulating isperformed by a light modulation unit of the CI device;

(b) generating an electrical signal representing intensity of themodulated light stream as a function of time, wherein the electricalsignal is generated by a light sensing device of the CI device;

(c) acquiring samples of the electrical signal, wherein the acquiredsamples include at least one sample for each of the spatial patterns;

(d) constructing an image using the sequence of spatial patterns and theacquired samples;

(e) displaying the image on a display;

(f) changing a focus setting of the CI device based on user input.

2.28 The method of paragraph 2.27, wherein the focus setting of the CIdevice is an optical distance between an optical subsystem of the CIdevice and the light modulation unit.

2.29 The method of paragraph 2.27, wherein the focus setting of the CIdevice is a focus setting of an optical subsystem of the CI device.

2.30 The method of paragraph 2.27, further comprising: performing (a)through (f) one or more additional times.

2B.1 A method for focusing a compressive imaging (CI) device, the methodcomprising:

(a) modulating an incident light stream with calibration patterns toobtain a modulated light stream, wherein said modulating is performed bya light modulation unit of the CI device, wherein the light modulationunit includes an array of light modulating elements;

(b) an array of light sensing elements of the CI device receiving themodulated light stream, where the light sensing elements receiverespective spatial portions of the modulated light stream and generaterespective electrical signals, wherein each electrical signal representsintensity of the respective spatial portion as a function of time,wherein the light sensing elements correspond to respectivenon-overlapping regions within the array of light modulating elements,wherein each of the calibration patterns specifies that the lightmodulating elements inside a respective one of the regions are to be setto an off state and specifies that at least a subset of the lightmodulating elements outside the respective region are to be set to an onstate;

(c) for each of the calibration patterns, acquiring a respective groupof samples from the array of light sensing elements, wherein each of thesample groups includes at least one sample of each of the electricalsignals;

(d) determining spillover values corresponding respectively to thecalibration patterns, wherein each of the spillover values is determinedbased on the sample group corresponding to the respective calibrationpattern, wherein each of the spillover values indicates an extent towhich modulated light from outside the respective region of the array oflight modulating elements reaches the light sensing elementcorresponding to the respective region.

(e) computing a composite spillover value for the array of light sensingelements based on the spillover values.

2B.2 The method of paragraph 2B.1, further comprising: directing anactuator to change an optical distance between the light modulation unitand the array of light sensing elements based on data including thecomposite spillover value.

2B.3 The method of paragraph 2B.1, further comprising repeatedlyperforming a set of operations that includes: changing an opticaldistance between the light modulation unit and the array of lightsensing elements; and performing (a) through (e), wherein the set ofoperations is repeated until the composite spillover value becomes lessthan a threshold value.

2B.4 The method of paragraph 2B.1, further comprising: directing anactuator to change an angle of orientation of the array of lightmodulating elements based on data including the composite spillovervalue.

2B.5 The method of paragraph 2B.1, further comprising repeatedlyperforming a set of operations that includes: changing an angle oforientation of the array of light sensing elements; and performing (a)through (e), wherein the set of operations is repeated until thecomposite spillover value becomes less than a threshold value.

2B.6 The method of paragraph 2B.1, wherein each of the calibrationpatterns specifies that all the light modulating elements outside therespective region are to take the ON state.

2B.7 The method of paragraph 2B.1, further comprising: directing anactuator to change an optical distance between the light modulation unitand the array of light sensing elements according to a firstdisplacement value;

performing (a) through (e) a second time, to obtain an additionalcomposite spillover value;

computing a second displacement value for the optical distance betweenthe light modulation unit and the array of light sensing elements basedon data including

the composite spillover value, the additional composite spillover valueand the first displacement value.

2B.8 The method of paragraph 2B.7, further comprising: directing theactuator to change the optical distance between the light modulationunit and the array of light sensing elements according to the seconddisplacement value.

2B.9 The method of paragraph 2B.1, further comprising: directing anactuator to change an angle of orientation of the array of light sensingelements according to a first displacement value;

performing (a) through (e) a second time, to obtain an additionalcomposite spillover value;

computing a second displacement value for the angular orientation basedon data including the composite spillover value, the additionalcomposite spillover value and the first displacement value.

2B.10 The method of paragraph 2B.9, further comprising: directing theactuator to change the angle of orientation of the array of lightsensing elements according to the second displacement value.

2B.11 The method of paragraph 2B.1, further comprising: directing anactuator to change an optical distance between the light modulation unitand the array of light sensing elements through a range of distances;

performing (a) through (e) for each of a plurality of optical distanceswithin the range in order to obtain a corresponding plurality ofcomposite spillover values;

determining an optimal value for the optical distance based on ananalysis of data including the plurality of composite spillover values.

2B.12 The method of paragraph 2B.11, further comprising: directing theactuator to change the optical distance to the optimal value.

2B.13 The method of paragraph 2B.1, further comprising: directing anactuator to change an orientation angle of the array of light sensingelements through a range of angles; performing (a) through (e) for eachof a plurality of angles within the range in order to obtain acorresponding plurality of composite spillover values; determining anoptimal value for the angle based on an analysis of data including theplurality of composite spillover values.

2B.14 The method of paragraph 2B.13, further comprising: directing theactuator to change the angular orientation to the optimal value.

2B.15 The method of paragraph 2B.1, wherein the number of thecalibration patterns is four.

2B.16 A method for focusing a compressive imaging (CI) device, themethod comprising:

(a) modulating an incident light stream with a calibration pattern toobtain a modulated light stream, wherein said modulating is performed bya light modulation unit of the CI device, wherein the light modulationunit includes an array of light modulating elements;

(b) an array of light sensing elements of the CI device receiving themodulated light stream, where the light sensing elements receiverespective spatial portions of the modulated light stream and generaterespective electrical signals, wherein each electrical signal representsintensity of the respective spatial portion as a function of time,wherein the light sensing elements correspond to respectivenon-overlapping regions within the array of light modulating elements,wherein the calibration pattern specifies that the light modulatingelements inside a first one of the regions are to be set to an off stateand specifies that at least a subset of the light modulating elementsoutside the first region are to be set to an on state, wherein the firstregion corresponds to a first of the light sensing elements;

(c) acquiring samples of the electrical signal generated by the firstlight sensing element in response to said modulating the incident lightstream with the calibration pattern, wherein each of the samplesindicates an extent to which modulated light from outside the firstregion of the array of light modulating elements reaches the first lightsensing element.

2B.17 A method for focusing a compressive imaging (CI) device, themethod comprising:

(a) modulating an incident light stream with calibration patterns toobtain a modulated light stream, wherein said modulating is performed bya light modulation unit of the CI device, wherein the light modulationunit includes an array of light modulating elements;

(b) an array of light sensing elements of the CI device receiving themodulated light stream, wherein each of the light sensing elementsreceives a respective spatial portion of the modulated light stream andgenerates a respective electrical signal representing intensity of therespective spatial portion as a function of time, wherein thecalibration patterns correspond to respective regions of the array oflight modulating elements and correspond to respective ones of the lightsensing elements, wherein each of the calibration patterns specifiesthat at least a subset of the light modulating elements inside therespective region are to be set to an on state and specifies that thelight modulating elements outside the respective region are to be set toan off state;

(c) for each of the calibration patterns, acquiring a respective groupof samples from the array of light sensing elements, wherein each of thesample groups includes at least one sample of each of the electricalsignals;

(d) computing spillover values corresponding respectively to thecalibration patterns, wherein each of the spillover values is computedbased on the sample group corresponding to the respective calibrationpattern, wherein each of the spillover values indicates an extent towhich modulated light from the respective region of the array of lightmodulating elements reaches light sensing elements of the array otherthan the respective light sensing element; and

(e) computing a composite spillover value for the array of light sensingelements based on the spillover values.

2B.18 A method for focusing a compressive imaging (CI) device, themethod comprising:

(a) modulating an incident light stream with a calibration pattern toobtain a modulated light stream, wherein said modulating is performed bya light modulation unit of the CI device, wherein the light modulationunit includes an array of light modulating elements;

(b) an array of light sensing elements of the CI device receiving themodulated light stream, wherein each of the light sensing elementsreceives a respective spatial portion of the modulated light stream andgenerates a respective electrical signal representing intensity of therespective spatial portion as a function of time, wherein thecalibration pattern correspond to a region of the array of lightmodulating elements and to one of the light sensing elements, whereinthe calibration pattern specifies that at least a subset of the lightmodulating elements within the region are to be set to the on state, andspecifies that the light modulating elements outside the region are tobe set to the off state;

(c) acquiring a group of samples corresponding to the calibrationpattern, wherein the sample group includes at least one sample of eachof the electrical signals;

(d) computing a spillover value for the calibration pattern, wherein thespillover value is computed based on the sample group and indicates anextent to which modulated light from the region of the array of lightmodulating elements reaches light sensing elements other than thecorresponding light sensing element.

2B.19 The method of paragraph 2B.18, wherein the region is located at acenter of the array of light modulating elements.

2B.20. A method comprising:

focusing a compressive imaging (CI) device, wherein the CI deviceincludes a light modulation unit and an array of light sensing elements,wherein the light modulation unit includes an array of L lightmodulating elements, wherein the light modulation unit is configured tomodulate an incident light stream with a sequence of spatial patterns toproduce a modulated light stream, wherein each of the L light sensingelements is configured to generate a respective electrical signalrepresenting intensity of a respective spatial portion of the modulatedlight stream as a function of time, wherein said focusing the CI deviceincludes:

(a) injecting p calibration patterns into the sequence of spatialpatterns so that the light modulation unit modulates the incident lightstream with each of the p calibration patterns, wherein p is a positiveinteger that is less than or equal to L, wherein the p calibrationpatterns correspond respectively to p non-overlapping regions of thearray of light modulating elements, wherein the p non-overlappingregions correspond respectively to p of the L light sensing elements,wherein each of the p calibration patterns specifies that at least asubset of the light modulating elements within the respective region areto take an ON state and that light modulating elements outside therespective region are to take an OFF state;

(b) for each of the p calibration patterns, acquiring a correspondinggroup of samples from the array of light sensing elements, wherein eachof the p sample groups includes at least one sample of each of the Lelectrical signals;

(c) computing a set of p spillover values corresponding respectively tothe p calibration patterns, wherein each of the p spillover values iscomputed based on the corresponding sample group and indicates an extentto which modulated light from the respective region of the array oflight modulating elements reaches light sensing elements of the arrayother than the respective light sensing element; and

(d) computing a composite spillover value for the array of light sensingelements based on the set of p spillover values.

2C.1 A method for determining focus-related information for acompressive imaging (CI) device, the method comprising:

(a) modulating an incident light stream with a sequence of spatialpatterns to obtain a modulated light stream, wherein said modulating isperformed by a light modulation unit of the CI device, wherein the lightmodulation unit includes an array of light modulating elements;

(b) an array of light sensing elements of the CI device receiving themodulated light stream, wherein the light sensing elements receiverespective spatial portions of the modulated light stream and generaterespective electrical signals, wherein each of the electrical signalsrepresents intensity of the respective spatial portion as a function oftime, wherein the light sensing elements correspond to respectivenon-overlapping regions of the array of light modulating elements;

(c) acquiring measurements from the array of light sensing elements,wherein the measurements include sample sets that correspondrespectively to the light sensing elements, wherein each sample setincludes samples of the electrical signal generated by the respectivelight sensing element, with the samples including at least one samplefor each of the spatial patterns (e.g., exactly one); (in the text,explain that you might only acquire one set of samples from one sensor)

(d) computing a focus indicator value based on two or more noise valuesfor two or more respective ones of the light sensing devices, whereinthe focus indicator value indicates an extent to which the modulatedlight stream is in focus at the sensor array, wherein each of the two ormore noise values is computed by: constructing a respective sub-imagebased on the sample set of the respective light sensing element and alsoon a restriction of the spatial patterns to the region of the array oflight modulating elements corresponding to the respective light sensingelement; and computing an amount of noise present in the respectivesub-image.

2C.2 The method of paragraph 2C.1, further comprising: determining adisplacement value for the array of light sensing elements based on dataincluding the focus indicator value; and directing the array of lightsensing elements to be translated according to the displacement value.

2C.3 The method of paragraph 2C.1, further comprising: determining anangular displacement value for the array of light sensing elements basedon data including the focus indicator value; and directing the array oflight sensing elements to be rotated according to the angulardisplacement value.

2C.4 The method of paragraph 2C.1, further comprising: changing anoptical distance between the light modulation unit and the array oflight sensing elements; and performing said changing and the operations(a) through (d) one or more times until the focus indicator value isoptimized.

2C.5 The method of paragraph 2C.1, further comprising: changing anangular orientation of the array of light sensing elements; andperforming said changing and the operations (a) through (d) one or moretimes until the focus indicator value is optimized.

2C.6. A method for determining focus-related information for acompressive imaging (CI) device, the method comprising:

(a) modulating an incident light stream with a sequence of spatialpatterns to obtain a modulated light stream, wherein said modulating isperformed by a light modulation unit of the CI device, wherein the lightmodulation unit includes an array of light modulating elements;

(b) an array of light sensing elements of the CI device receiving themodulated light stream, wherein the light sensing elements receiverespective spatial portions of the modulated light stream and generaterespective electrical signals, wherein each of the electrical signalsrepresents intensity of the respective spatial portion as a function oftime, wherein the light sensing elements correspond to respectivenon-overlapping regions of the array of light modulating elements;

(c) acquiring a set of samples from a first of the light sensingelements, wherein the sample set includes samples of the electricalsignal generated by the first light sensing element, with the samplesincluding at least one sample for each of the spatial patterns;

(d) computing a focus indicator value for the first light sensingdevice, wherein the focus indicator value indicates an extent to whichthe modulated light stream is in focus at the sensor array, wherein thefocus indicator value is computed by: constructing a sub-image based onthe sample set and on a restriction of the spatial patterns to a theregion of the array of light modulating elements corresponding to thefirst light sensing element; and computing an amount of noise present inthe sub-image.

2D.1. A method for determining focus-related information for acompressive imaging (CI) device, the method comprising:

(a) modulating an incident light stream with a sequence of spatialpatterns to obtain a modulated light stream, wherein said modulating isperformed by a light modulation unit of the CI device, wherein the lightmodulation unit includes an array of light modulating elements;

(b) an array of light sensing elements of the CI device receiving themodulated light stream, wherein the light sensing elements receiverespective spatial portions of the modulated light stream and generaterespective electrical signals, wherein each of the electrical signalsrepresents intensity of the respective spatial portion as a function oftime, wherein the light sensing elements correspond to respectivenon-overlapping regions on the array of light modulating elements;

(c) acquiring measurements from the array of light sensing elements,wherein the measurements include sample sets that correspondrespectively to the light sensing elements, wherein each sample setincludes samples of the electrical signal generated by the respectivelight sensing element, with the samples including at least one samplefor each of the spatial patterns;

(d) computing a focus indicator value based on two or more spillovervalues for two or more respective ones of the light sensing devices,wherein the focus indicator value indicates an extent to which themodulated light stream is in focus at the sensor array, wherein each ofthe two or more spillover values is computed by: constructing arespective image based on the sample set of the respective light sensingelement and also on the spatial patterns; and summing pixels in theimage outside the region corresponding to the respective light sensingelement.

2D.2. A method for determining focus-related information for acompressive imaging (CI) device, the method comprising:

(a) modulating an incident light stream with a sequence of spatialpatterns to obtain a modulated light stream, wherein said modulating isperformed by a light modulation unit of the CI device, wherein the lightmodulation unit includes an array of light modulating elements;

(b) an array of light sensing elements of the CI device receiving themodulated light stream, wherein the light sensing elements receiverespective spatial portions of the modulated light stream and generaterespective electrical signals, wherein each of the electrical signalsrepresents intensity of the respective spatial portion as a function oftime, wherein the light sensing elements correspond to respectivenon-overlapping regions on the array of light modulating elements;

(c) acquiring measurements from the array of light sensing elements,wherein the measurements include sample sets that correspondrespectively to the light sensing elements, wherein each sample setincludes samples of the electrical signal generated by the respectivelight sensing element, with the samples including at least one samplefor each of the spatial patterns;

(d) computing a spillover value for a first of the light sensingdevices, wherein the spillover value indicates an extent to which themodulated light stream is in focus at the sensor array, wherein thespillover value is computed by: constructing an image based on thesample set corresponding to the first light sensing element and also onthe spatial patterns; and summing pixels in the image outside the regioncorresponding to the first light sensing element.

3.1 A method for focusing a compressive imaging (CI) device, the methodcomprising:

(a) supplying a sequence of high-frequency spatial patterns to a lightmodulation unit of the CI device, wherein the light modulation unitincludes an array of light modulating elements, wherein each of thehigh-frequency spatial patterns is a spatial pattern whose AC componentincludes only spatial frequencies greater than or equal to a cutofffrequency;

(b) the light modulation unit modulating an incident light stream withthe sequence of high-frequency spatial patterns to produce a modulatedlight stream;

(c) a light sensing device of the CI device receiving the modulatedlight stream and acquiring a sequence of samples representing anintensity of the modulated light stream as a function of time, whereinthe sequence of samples includes at least one sample for each of thehigh-frequency spatial patterns;

(d) computing a focus indicator value based on the sequence of samples,wherein the focus indicator value indicates an extent to which theincident light stream is in focus at the array of light modulatingelements.

3.2 The method of paragraph 3.1, wherein each of the high-frequencyspatial patterns corresponds to spatial sinusoid whose spatial frequencyis greater than or equal to the cutoff frequency.

3.3 The method of paragraph 3.1, wherein the cutoff-frequency isprogrammable.

3.4 The method of paragraph 3.1, wherein the CI device includes anoptical subsystem that receives and operates on the incident lightstream prior to the incident light stream arriving at the lightmodulation unit.

3.5 The method of paragraph 3.4, further comprising: (e) changing afocus setting of the optical subsystem based on data including the focusindicator value.

3.6 The method of paragraph 3.4, further comprising repeatedlyperforming a set of operations including: changing a focus setting ofthe optical subsystem; and performing (a), (b), (c) and (d), wherein theset of operations is repeated until the focus indicator value becomesless than a threshold value.

3.7 The method of paragraph 3.4, further comprising: changing an opticaldistance between the light modulation unit and the optical subsystembased on data including the focus information.

3.8 The method of paragraph 3.4, further comprising repeatedlyperforming a set of operations including: changing an optical distancebetween the light modulation unit and the optical subsystem; andperforming (a), (b), (c) and (d), wherein the set of operations isrepeated until the focus indicator value becomes less than a thresholdvalue.

3.9 The method of paragraph 3.4, further comprising: directing anactuator to change an optical distance between the optical subsystem andthe light modulation unit according to a first displacement value;performing (a) through (d) a second time, to obtain an additional focusindicator value for an additional sequence of samples; and computing asecond displacement value for the optical distance based on dataincluding the focus indicator value for the sequence of samples, thefocus indicator value for the additional sequence of samples and thefirst displacement value.

3.10 The method of paragraph 3.9, further comprising: directing theactuator to change the optical distance based on the second displacementvalue.

3.11 The method of paragraph 3.4, further comprising: directing anactuator to change a focus setting of the optical subsystem according toa first displacement value; performing (a) through (d) a second time, toobtain an additional focus indicator value for an additional sequence ofsamples; and computing a second displacement value for the focus settingbased on data including the focus indicator value for the sequence ofsamples, the focus indicator value for the additional sequence ofsamples and the first displacement value.

3.12 The method of paragraph 3.11, further comprising: directing theactuator to change the focus setting of the optical subsystem accordingto the second displacement value.

3.13 The method of paragraph 3.1, wherein said computing the focusindicator value includes: removing a temporal DC component of thesequence of samples to obtain an AC sequence of samples; and computingan average magnitude of the AC sequence of samples.

4.1 A system comprising:

a light modulation unit configured to receive an incident light streamfrom an optical input path, wherein the light modulation unit includesan array of mirrors configured to receive respective portions of theincident light stream, wherein each of the mirrors is configured to (a)reflect the respective portion of the incident light stream onto a firstoptical path when the mirror is in a first orientation state and (b)reflect the respective portion of the incident light stream onto asecond optical path when the mirror is in a second orientation state;

a light sensing device configured to receive a first light streamcomprising the portions of the incident light stream that are reflectedonto the first optical path by mirrors in the first orientation state,wherein the first light sensing device is configured to generate asequence of samples representing intensity of the first light stream asfunction of time;

a time-of-flight determining device (TDD) configured to: transmit alight pulse onto the second optical path so that the light pulse isreflected onto the input optical path and out of the system by mirrorsthat are in the second orientation state; receive a second light streamcomprising the portions of the incident light stream that are reflectedonto the second optical path by the mirrors in the second orientationstate; detect a reflected pulse occurring in the second light stream;and determine a time of flight between the transmission of the lightpulse and detection of the reflected pulse;

a processing unit configured to determine a range from the system to anexternal object based on the time of flight.

4.2 The system of paragraph 4.1, further comprising: an opticalsubsystem positioned along the input optical path and configured tooperate on the incident light stream before it is provided to the lightmodulation unit.

4.3 The system of paragraph 4.2, further comprising: a first actuatorconfigured to adjust a first optical distance between the opticalsubsystem and the light modulation unit, wherein the processing unit isconfigured to compute a first adjustment value for the first opticaldistance based on the determined range and to direct the first actuatorto adjust the first optical distance according to the first adjustmentvalue.

4.4 The system of paragraph 4.3, further comprising: a second actuatorconfigured to adjust a second optical distance between the lightmodulation unit and the light sensing device, wherein the processingunit is configured to compute a second adjustment value for the secondoptical distance based on the first adjustment value and to direct thesecond actuator to adjust the second optical distance according to thesecond adjustment value.

4.5 The system of paragraph 4.3, further comprising: a second actuatorconfigured to adjust an angular orientation of the light sensing device,wherein the processing unit is configured to compute an angle adjustmentvalue for the angular orientation based on the first adjustment valueand to direct the second actuator to adjust the angular orientationaccording to the angle adjustment value.

4.6 The system of paragraph 4.2, further comprising: a first actuatorconfigured to adjust a focus setting of the optical subsystem, whereinthe processing unit is configured to direct the first actuator to adjustsaid focus setting based on the determined range.

4.7 The system of paragraph 4.1, wherein the TDD includes a transmitter,a detector, and a timing subsystem, wherein the transmitter isconfigured to generate the light pulse, wherein the detector isconfigured to receive the second light stream and to detect thereflected pulse, wherein the timing subsystem is configured to determinethe time of flight between said transmission and said detection.

4.8 The system of paragraph 4.7, wherein the transmitter includes alaser light source.

4.9 The system of paragraph 4.1, further comprising: a control unitconfigured to drive the array of mirrors through a sequence of spatialpatterns, wherein the TDD is configured to transmit the light pulse anddetect the reflected pulse within a period of time corresponding to asingle one of the spatial patterns.

4.10 The system of paragraph 4.1, wherein the sequence of samplescorresponds to M of the spatial patterns, wherein the sequence ofsamples is usable to construct an N-pixel image, wherein M is less thanN.

4.11 The system of paragraph 4.1, wherein the TDD is configured todetermine a Doppler shift between the light pulse and the reflectedpulse, wherein the processing unit is configured to compute a rangevelocity based on the Doppler shift.

4.12 A system comprising:

a light modulation unit configured to receive an incident light streamfrom an optical input path, wherein the light modulation unit includesan array of mirrors configured to receive respective portions of theincident light stream, wherein each of the mirrors is configured to (a)reflect the respective portion of the incident light stream onto a firstoptical path when the mirror is in a first orientation state and (b)reflect the respective portion of the incident light stream onto asecond optical path when the mirror is in a second orientation state;

a light sensing device configured to receive a first light streamcomprising the portions of the incident light stream that are reflectedonto the first optical path by mirrors in the first orientation state,wherein the light sensing device is configured to generate a sequence ofsamples representing intensity of the first light stream as function oftime;

a range finder configured to: transmit a light signal onto the secondoptical path so that the light signal is reflected onto the inputoptical path and out of the system by mirrors that are in the secondorientation state; receive a second light stream comprising the portionsof the incident light stream that are reflected onto the second opticalpath by the mirrors in the second orientation state, wherein the secondlight stream includes a reflected light signal due to reflection of thetransmitted light signal from an external object; and determine a rangeto the external object based on the transmitted light signal and thereflected light signal.

4.13 The system of paragraph 4.12, further comprising: a processingunit; and an optical subsystem positioned along the input optical pathand configured to operate on the incident light stream before it isprovided to the light modulation unit.

4.14 The system of paragraph 4.13, further comprising: a first actuatorconfigured to adjust a first optical distance between the opticalsubsystem and the light modulation unit, wherein the processing unit isconfigured to compute an optimal value for the first optical distancebased on the determined range and to direct the first actuator to setthe first optical distance to the optimal value for the first opticaldistance.

4.15 The system of paragraph 4.14, further comprising: a second actuatorconfigured to adjust a second optical distance between the lightmodulation unit and the light sensing device, wherein the processingunit is configured to compute an optimal value for the second opticaldistance based on the optimal value for the first optical distance andto direct the second actuator to set the second optical distance to theoptimal value for the second optical distance.

4.16 The system of paragraph 4.13, further comprising: a first actuatorconfigured to adjust a first optical distance between the opticalsubsystem and the light modulation unit, wherein the processing unit isconfigured to compute a first adjustment value for the first opticaldistance based on the determined range and to direct the first actuatorto adjust the first optical distance according to the first adjustmentvalue.

4.17 The system of paragraph 4.16, further comprising: a second actuatorconfigured to adjust a second optical distance between the lightmodulation unit and the light sensing device, wherein the processingunit is configured to compute a second adjustment value for the secondoptical distance based on the first adjustment value and to direct thesecond actuator to adjust the second optical distance according to thesecond adjustment value.

4.18 The system of paragraph 4.16, further comprising: a second actuatorconfigured to adjust an angular orientation of the light sensing device,wherein the processing unit is configured to compute an angle adjustmentvalue for the angular orientation based on the first adjustment valueand to direct the second actuator to adjust the angular orientationaccording to the angle adjustment value.

4.19 The system of paragraph 4.13, further comprising: a first actuatorconfigured to adjust a focus setting of the optical subsystem, whereinthe processing unit is configured to: compute an optimal value for thefocus setting based on the determined range; and direct the firstactuator to set the focus setting to the optimal value for the focussetting.

4.20 The system of paragraph 4.19, further comprising: a second actuatorconfigured to adjust an optical distance between the light modulationunit and the light sensing device, wherein the processing unit isconfigured to: compute an optimal value for the optical distance basedon the optimal value for the focus setting and to direct the secondactuator to set the optical distance to the optimal value for theoptical distance.

4.21 The system of paragraph 4.13, further comprising: a first actuatorconfigured to adjust a focus setting of the optical subsystem, whereinthe processing unit is configured to direct the first actuator to adjustsaid focus setting based on the determined range.

4.22 The system of paragraph 4.12, wherein the range finder includes alaser light source.

4.23 The system of paragraph 4.12, further comprising: a control unitconfigured to drive the array of mirrors through a sequence of spatialpatterns, wherein the range finder is configured to transmit the lightsignal and receive the reflected light signal while the array of mirrorsis being driven through the sequence of spatial patterns.

4.24 The system of paragraph 4.23, wherein the sequence of samplescorresponds to M of the spatial patterns, wherein the sequence ofsamples is usable to construct an N-pixel image, wherein M is less thanN.

4.25 The system of paragraph 4.12, wherein the range finder isconfigured to determine a Doppler shift between the transmitted lightsignal and the reflected light signal, wherein the processing unit isconfigured to compute a range velocity based on the Doppler shift.

5.1 A system comprising:

a light modulation unit configured to receive an incident light streamfrom an optical input path, wherein the light modulation unit includesan array of mirrors configured to receive respective portions of theincident light stream, wherein each of the mirrors is configured to (a)reflect the respective portion of the incident light stream onto a firstoptical path when the mirror is in a first orientation state and (b)reflect the respective portion of the incident light stream onto asecond optical path when the mirror is in a second orientation state;

a first bandpass filter configured to restrict a first light stream to afirst wavelength range in order to produce a restricted light stream,wherein the first light stream comprises the portions of the incidentlight stream that are reflected onto the first optical path by mirrorsin the first orientation state;

a light sensing device configured to receive the restricted light streamand generate a sequence of samples representing intensity of therestricted light stream as a function of time;

a second bandpass filter configured to restrict a second light stream toa visible wavelength range in order to obtain a visible light stream,wherein the second light stream comprises the portions of the incidentlight stream that are reflected onto the second optical path by themirrors in the second orientation state;

an image sensor configured to acquire a sequence of images of thevisible light stream, wherein the image sensor includes an array oflight detecting elements.

5.2 The system of paragraph 5.1, wherein the first wavelength range andthe visible wavelength range are non-overlapping (or non-identical).

5.3 The system of paragraph 5.1, wherein the first bandpass filter isconfigured so that the first wavelength range is adjustable (orselectable).

5.4 The system of paragraph 5.1, further comprising: an actuatorconfigured to adjust an optical distance between the light modulationunit and the light sensing device, wherein the first bandpass filter isconfigured so that the first wavelength range is selectable, wherein theprocessing unit is configured to: receive user input specifying aselection for the first wavelength range; determine an adjustment valuefor the optical distance based on the specified selection for the firstwavelength range; and direct the actuator to adjust the optical distancebased on the determined adjustment value.

5.5 The system of paragraph 5.1, further comprising: an opticalsubsystem configured to receive and operate on the incident light streamprior to the incident light stream arriving at the light modulationunit; a display; and a processing unit configured to display thesequence of images on the display.

5.6 The system of paragraph 5.5, further comprising: a first actuatorconfigured to adjust a focus setting of the optical subsystem, whereinthe processing unit is configured to: receive user input specifying afocus adjustment; and direct the first actuator to adjust the focussetting of the optical subsystem based on the user-specified focusadjustment.

5.7 The system of paragraph 5.5, further comprising: an actuatorconfigured to adjust a pre-modulation optical distance between theoptical subsystem and the light modulation unit, wherein the processingunit is configured to: receive user input specifying a focus adjustment;and direct the actuator to adjust the pre-modulation optical distancebased on the user-specified focus adjustment.

5.8 The system of paragraph 5.7, wherein the processing unit is furtherconfigured to: compute a first adjustment value for a firstpost-modulation optical distance between the light modulation unit andthe image sensor based on the user-specified focus adjustment; computinga second adjustment value for a second post-modulation optical distancebetween the light modulation unit and the light sensing device based onthe first adjustment value; direct a first additional actuator to adjustthe first post-modulation optical distance based on the first adjustmentvalue; and direct a second additional actuator to adjust the secondpost-modulation optical distance based on the second adjustment value.

5.9 The system of paragraph 5.5, further comprising: a first actuatorconfigured to adjust a first optical distance between the lightmodulation unit and the image sensor.

5.10 The system of paragraph 5.9, further comprising: a second actuatorconfigured to adjust a second optical distance between the lightmodulation unit and the light sensing device, wherein the processingunit is configured to: receive user input specifying a focus adjustmentfor the first optical distance; direct the first actuator to adjust thefirst optical distance based on the user-specified focus adjustment;determine an adjustment value for the second optical distance based onthe user-specified focus

adjustment; and direct the second actuator to adjust the second opticaldistance based on the determined adjustment value.

5.11 The system of paragraph 5.1, further comprising: a control unitconfigured to drive the array of mirrors through a sequence of spatialpatterns, wherein the sequence of spatial patterns includes a firstsubsequence, wherein each spatial pattern of the first subsequencespecifies that all the mirrors be in the second orientation state.

5.12 The system of paragraph 5.1, further comprising: an opticalsubsystem configured to receive and operate on the incident light streamprior to the incident light stream arriving at the light modulationunit; a processing unit configured to compute a focus indicator valuebased on at least one of the images, wherein the focus indicator valueindicates an extent to which the visible light stream is in focus at theimage sensor.

5.13 The system of paragraph 5.12, further comprising: a first actuatorconfigured to adjust a focus setting of the optical subsystem, whereinthe processing unit is configured to direct the first actuator to adjustthe focus setting of the optical subsystem based on data including thefocus indicator value.

5.14 The system of paragraph 5.12, further comprising: a first actuatorconfigured to adjust a first optical distance between the opticalsubsystem and the light modulation unit, wherein the processing unit isconfigured to: compute a first adjustment value for the first opticaldistance based on data including the focus indicator value; and directthe first actuator to adjust the first optical distance based on thefirst adjustment value.

5.15 The system of paragraph 5.14, wherein the processing unit isfurther configured to: compute a second adjustment value for a secondoptical distance between the light modulation unit and the image sensorbased on the first adjustment value; compute a third adjustment valuefor a third optical distance between the light modulation unit and thelight sensing device based on the first adjustment value; direct asecond actuator to adjust the second optical distance based on thesecond adjustment value; direct a third actuator to adjust the thirdoptical distance based on the third adjustment value.

5.16 The system of paragraph 5.12, further comprising: a first actuatorconfigured to adjust a first optical distance between the lightmodulation unit and the image sensor.

5.17 The system of paragraph 5.16, a second actuator configured toadjust a second optical distance between the light modulation unit andthe light sensing device, wherein the processing unit is configured to:compute a first adjustment value for the first optical distance based ondata including the focus indicator value; compute a second adjustmentvalue for the second optical distance based on the first adjustmentvalue; direct the first actuator to adjust the first optical distancebased on the first adjustment value; and direct the second actuator toadjust the second optical distance based on the second adjustment value.

5.18 The system of paragraph 5.1, further comprising: an opticalsubsystem configured to receive and operate on the incident light streamprior to the incident light stream arriving at the light modulationunit; a first actuator configured to adjust a focus setting of theoptical subsystem; a control unit configured to drive the array ofmirrors through a sequence of spatial patterns; a processing unitconfigured to operate on a first subset of the sequence of imagesacquired by the image sensor to determine focus indicator values,wherein each of focus indicator values is determined based on one ormore of the images of the first subset, wherein each of the focusindicator values indicates an extent to which the visible light streamis in focus at the image sensor.

5.19 The system of paragraph 5.18, wherein the processing unit isconfigured to drive the first actuator based on the focus indicatorvalues in order to locate an improved value of the focus setting.

5.20 The system of paragraph 5.18, wherein the spatial patterns includeone-dimensional patterns, wherein each of the one-dimensional patternsis restricted to a corresponding one-dimensional path on the array ofthe mirrors.

5.21 The system of paragraph 5.1, further comprising:

a first actuator configured to adjust a first optical path lengthbetween the light modulation unit and the image sensor;

a second actuator configured to adjust a second optical path lengthbetween the light modulation unit and the light sensing device;

a control unit configured to drive the array of mirrors through asequence of spatial patterns, wherein the sequence of spatial patternsincludes a plurality of repetitions of a collection of one or morecalibration patterns, wherein the one or more calibration patternscorrespond respectively to one or more subsets of the array of mirrors,wherein each of the one or more subsets corresponds to a respectivesubset of the array of light detecting elements, wherein the sequence ofimages acquired by the image sensor includes an image for each of thespatial patterns; and

a processing unit configured to compute a sequence of spillover values,wherein each of the spillover values corresponds to respective one ofthe plurality of repetitions of the set of one or more calibrationpatterns and is computed based on one or more of the images thatcorrespond to the respective repetition, wherein each spillover valueindicates an extent to which modulated light from the one or moresubsets of the array of mirrors fail to reach their respective subsetsof the array of light detecting elements.

5.22 The system of paragraph 5.21, wherein the processing unit isconfigured to: drive the first actuator based on the spillover values inorder to determine an improved value for the first optical path length;compute an improved value for the second optical path length based onthe improved value for the first optical path length; and direct thesecond actuator to adjust the second optical path length based on theimproved value for the second optical path length.

5B.1 A system comprising:

a first optical subsystem configured to receive an incident light streamand to split the incident light stream into a first light stream and asecond light stream;

a light modulation unit configured to modulate the first light streamwith a sequence of spatial patterns to obtain a modulated light stream;

a first light sensing device configured to receive the modulated lightstream and to generate a first sequence of samples representingintensity of the modulated light stream as a function of time;

an image sensor configured to receive the second light stream and toacquire a sequence of images of the second light stream, wherein theimage sensor includes an array of light detecting elements.

5B.2 The system of paragraph 5B.1, further comprising: a first bandpassfilter configured to receive and the modulated light stream and torestrict the modulated light stream to a first wavelength range, whereinthe first light sensing device is configured to receive the modulatedlight stream as restricted to the first wavelength range.

5B.3 The system of paragraph 5B.2, wherein the first bandpass filter isconfigured so that the first wavelength range is adjustable.

5B.4 The system of paragraph 5B.2, further comprising an actuatorconfigured to adjust an optical distance between the light modulationunit and the first light sensing device, wherein the first bandpassfilter is configured so that the first wavelength range is selectable,wherein the processing unit is configured to: receive user inputspecifying a selection for the first wavelength range; determine a valuefor the optical distance based on the specified selection for the firstwavelength range; and direct the actuator to adjust the optical distancebased on the determined value.

5B.5 The system of paragraph 5B.1, further comprising a second bandpassfilter configured to restrict the second light stream to a visiblewavelength range, wherein the image sensor is configured to receive thesecond light stream as restricted to the visible wavelength range.

5B.6 The system of paragraph 5B.1, wherein the first light stream isrestricted to a first wavelength range and the second light stream isrestricted to a visible wavelength range, wherein the first wavelengthrange is not the same as the visible wavelength range.

5B.7 The system of paragraph 5B.1, further comprising: a second opticalsubsystem configured to receive and operate on the incident light streamprior to the incident light stream arriving at the first opticalsubsystem; a display; and a processing unit configured to display thesequence of images on the display.

5B.8 The system of paragraph 5B.7, wherein the processing unit isconfigured to: receive user input specifying a focus adjustment; andadjust a focus setting of the second optical subsystem based on theuser-specified focus adjustment.

5B.9 The system of paragraph 5B.8, wherein the processing unit isconfigured to: evaluate a first calibration function based on a currentvalue of the focus setting to obtain an second value of the focussetting, wherein the first calibration function relates optimal valuesof the focus setting for focusing the second light stream onto the imagesensor to optimal values of the focus setting for focusing the firstlight stream onto the light modulation unit; and change the focussetting of the second optical subsystem to the second value.

5B.10 The system of paragraph 5B.9, wherein the processing unit isfurther configured to: evaluate a second calibration function based onthe second value of the focus setting to obtain an optimal value for anoptical distance between the light modulation unit and the light sensingdevice; and change the optical distance to the optimal value.

5B.11 The system of paragraph 5B.7, wherein the processing unit isconfigured to: receive user input specifying a focus adjustment; andchange a first optical distance between the first optical subsystem andthe image sensor based on the user-specified focus adjustment.

5B.12 The system of paragraph 5B.11, wherein the processing unit isfurther configured to: evaluate a first calibration function based on acurrent value of the first optical distance to obtain a second value fora second optical distance between the first optical subsystem and thelight modulation unit, wherein the first calibration function relatesoptimal values of the first optical distance for focusing the secondlight stream onto the image sensor to optimal values of the secondoptical distance for focusing the first light stream onto the lightmodulation unit; and change the second optical distance to the secondvalue.

5B.13 The system of paragraph 5B.12, wherein the processing unit isfurther configured to: evaluate a second calibration function based onthe second value of the second optical distance to obtain an optimalvalue for a third optical distance between the light modulation unit andthe light sensing device; and change the third optical distance to theoptimal value.

5B.14 The system of paragraph 5B.1, further comprising: a second opticalsubsystem configured to receive and operate on the incident light streamprior to the incident light stream arriving at the first opticalsubsystem; and a processing unit configured to compute focus indicatorvalues for respective ones of the images, wherein each focus indicatorvalue indicates an extent to which the second light stream is in focusat the image sensor.

5B.15 The system of paragraph 5B.14, wherein the processing unit isconfigured to: change a focus setting of the second optical subsystemone or more times until the focus indicator value is optimized.

5B.16 The system of paragraph 5B.15, wherein the processing unit isconfigured to: evaluate a first calibration function based on a firstvalue of the focus setting to obtain a second value of the focussetting, wherein the first calibration function relates optimal valuesof the focus setting for focusing the second light stream onto the imagesensor to optimal values of the focus setting for focusing the firstlight stream onto the light modulation unit; and change the focussetting of the second optical subsystem to the second value.

5B.17 The system of paragraph 5B.16, wherein the processing unit isfurther configured to: evaluate a second calibration function based onthe second value of the focus setting to obtain an optimal value for anoptical distance between the light modulation unit and the light sensingdevice; and change the optical distance to the optimal value.

5B.18 The system of paragraph 5B.14, wherein the processing unit isconfigured to change a first optical distance between the first opticalsubsystem and the image sensor one or more times until the focusindicator value is optimized.

5B.19 The system of paragraph 5B.18, wherein the processing unit isconfigured to: evaluate a first calibration function based on a firstvalue of the first optical distance to obtain a second value for asecond optical distance between the first optical subsystem and thelight modulation unit, wherein the first calibration function relatesoptimal values of the first optical distance for focusing the secondlight stream onto the image sensor to optimal values of the secondoptical distance for focusing the first light stream onto the lightmodulation unit; and change the second optical distance to the secondvalue.

5B.20 The system of paragraph 5B.19, wherein the processing unit isfurther configured to: evaluate a second calibration function based onthe second value of the second optical distance to obtain an optimalvalue for a third optical distance between the light modulation unit andthe light sensing device; and change the third optical distance to theoptimal value.

5B.21 A system comprising:

a first optical subsystem configured to receive an incident light streamand to split the incident light stream into a first light stream and asecond light stream;

a light modulation unit configured to receive the first light stream andto modulate the first light stream, wherein the light modulation unitincludes an array of mirrors configured to receive respective portionsof the first light stream, wherein each of the mirrors is configured to(a) reflect the respective portion of the first light stream onto afirst optical path when the mirror is in a first orientation state and(b) reflect the respective portion of the first light stream away fromthe first optical path when the mirror is in a second orientation state;

a first light sensing device configured to receive a modulated lightstream from the first optical path, wherein the modulated light streamcomprises the portions of the first light stream that are reflected ontothe first optical path by mirrors in the first orientation state,wherein the first light sensing device is further configured to generatea first sequence of samples representing intensity of the modulatedlight stream as a function of time;

an image sensor configured to receive the second light stream and toacquire a sequence of images of the second light stream, wherein theimage sensor includes an array of light detecting elements.

5B.22 The system of paragraph 5B.21, wherein the first optical subsystemincludes a TIR prism pair, wherein a first prism of the TIR prism pairis configured to generate the first light stream by partiallytransmitting the incident light stream and generate the second lightstream by partially reflecting the incident light stream.

5B.23 The system of paragraph 5B.22, wherein a first surface of a firstprism of the TIR prism pair is coated with a dielectric coating in orderto restrict the second light stream to a visible wavelength range.

5B.24 The system of paragraph 5B.21, wherein each of the mirrors isconfigured to reflect the respective portion of the first light streamonto a second optical path when the mirror is in the second orientationstate, wherein the system further comprises a second light sensingdevice configured to receive a complementary modulated light stream fromthe second optical path, wherein the complementary modulated lightstream comprises the portions of the first light stream that arereflected onto the second optical path by mirrors in the secondorientation state, wherein the second light sensing device is furtherconfigured to generate a second sequence of samples representingintensity of the complementary modulated light stream.

Any of the various embodiments described herein may be combined to formcomposite embodiments. Furthermore, any of the various embodimentsdescribed in U.S. Provisional Application No. 61/372,826 and in U.S.patent application Ser. Nos. 13/193,553, 13/193,556 and 13/197,304 maybe combined with any of the various embodiments described herein to formcomposite embodiments.

Although the embodiments above have been described in considerabledetail, numerous variations and modifications will become apparent tothose skilled in the art once the above disclosure is fully appreciated.It is intended that the following claims be interpreted to embrace allsuch variations and modifications.

What is claimed is:
 1. A method for focusing a compressive imaging (CI)device, the method comprising: (a) supplying a sequence of spatialpatterns to a light modulation unit of the CI device; (b) the lightmodulation unit modulating an incident light stream with the sequence ofspatial patterns to produce a modulated light stream; (c) a lightsensing device of the CI device receiving the modulated light stream andacquiring a sequence of samples representing an intensity of themodulated light stream as a function of time, wherein the sequence ofsamples includes at least one sample for each of the spatial patterns;(d) constructing an image using the sequence of spatial patterns and theacquired samples; (e) computing a focus indicator value based on theimage, wherein the focus indicator value indicates an extent to whichthe incident light stream is in focus at the light modulation unit. 2.The method of claim 1, wherein each of the spatial patterns isrestricted to a subset of the array of light modulating elements.
 3. Themethod of claim 2, wherein the subset of the array of light modulatingelements is a convex region located at the center of the array of lightmodulating elements.
 4. The method of claim 1, wherein the CI deviceincludes an optical subsystem that receives and operates on the incidentlight stream prior to the incident light stream arriving at the lightmodulation unit.
 5. The method of claim 4, further comprising: changinga focus setting of the optical subsystem based on data including thefocus indicator value.
 6. The method of claim 5, further comprisingrepeatedly performing a set of operations that includes: changing afocus setting of the optical subsystem; and performing (a), (b), (c),(d) and (e), wherein the set of operations is repeated until the focusindicator value becomes less than a threshold value.
 7. The method ofclaim 4, further comprising: changing an optical distance between theoptical subsystem and the light modulation unit based on data includingthe focus indicator value.
 8. The method of claim 4, further comprisingrepeatedly performing a set of operations that includes: changing anoptical distance between the optical subsystem and the light modulationunit; and performing (a), (b), (c), (d) and (e), wherein the set ofoperations is repeated until the focus indicator value becomes less thana threshold value.
 9. The method of claim 8, further comprising:changing an optical distance between the light modulation unit and thelight sensing device based on a final distance between the opticalsubsystem and the light modulation unit after the focus indicator valuebecomes less than the threshold value.
 10. The method of claim 1,further comprising: changing an angle of orientation of the lightmodulation unit based on data including the focus indicator value. 11.The method of claim 4, further comprising: directing an actuator tochange an optical distance between the optical subsystem and the lightmodulation unit according to a first displacement value; performing (a)through (e) a second time, to obtain an additional image and a focusindicator value for the additional image; computing a seconddisplacement value for the optical distance based on data including thefocus indicator value for the image, the focus indicator value for theadditional image and the first displacement value.
 12. The method ofclaim 11, further comprising: directing the actuator to change theoptical distance according to the second displacement value.
 13. Themethod of claim 4, further comprising: directing an actuator to change afocus setting of the optical subsystem according to a first displacementvalue; performing (a) through (e) a second time, to obtain an additionalimage and a focus indicator value for the additional image; computing asecond displacement value for the focus setting based on data includingthe focus indicator value for the image, the focus indicator value forthe additional image and the first displacement value.
 14. The method ofclaim 13, further comprising: directing the actuator to change the focussetting of the optical subsystem according to the second displacementvalue.
 15. The method of claim 4, further comprising: directing anactuator to change an optical distance between the optical subsystem andthe light modulation unit through a range of distances; performing (a)through (e) for each of a plurality of distances within the range inorder to obtain a corresponding plurality of images and a correspondingplurality of focus indicator values; determining an optimal value forthe optical distance based on an analysis of data including theplurality of focus indicator values.
 16. The method of claim 15, furthercomprising: directing the actuator to change the optical distancebetween the optical subsystem and the light modulation unit to theoptimal value.
 17. The method of claim 4, further comprising: directingan actuator to change the focus setting of the optical subsystem througha range of settings; performing (a) through (e) for each of a pluralityof focus settings within the range in order to obtain a correspondingplurality of images and a corresponding plurality of focus indicatorvalues; and determining an optimal value for the focus setting based onan analysis of data including the plurality of focus indicator values.18. The method of claim 17, further comprising: directing the actuatorto change the focus setting to the optimal value.
 19. The method ofclaim 1, wherein the light sensing device includes a plurality of lightsensing elements, wherein each of the light sensing elements generates arespective electrical signal representing intensity of a respectivespatial portion of the modulated light stream, wherein said acquiringthe sequence of samples comprises adding the electrical signals toobtain a sum signal and sampling the sum signal.
 20. The method of claim1, wherein the light sensing device includes a plurality of lightsensing elements, wherein each of the light sensing elements generates arespective electrical signal representing intensity of a respectivespatial portion of the modulated light stream, wherein said acquiringthe sequence of samples comprises sampling each of the electricalsignals and adding the sampled versions of the electrical signals in thedigital domain.
 21. The method of claim 1, wherein the light sensingdevice includes only one light sensing element.
 22. A method forfocusing a compressive imaging (CI) device, the method comprising: (a)supplying a sequence of spatial patterns to a light modulation unit ofthe CI device; (b) the light modulation unit modulating an incidentlight stream with the sequence of spatial patterns to produce amodulated light stream; (c) acquiring a sequence of the frames from anarray of light sensing elements of the CI device, wherein each of thelight sensing elements receives a respective spatial portion of themodulated light stream, wherein each of the frames corresponds torespective one of the spatial patterns, wherein each of the framesincludes one sample from each of the light sensing elements; (d)determining a sequence of sum values, wherein each of the sum values isdetermined by adding the samples in a respective one of the frames ofthe acquired sequence; (e) constructing an image using the sequence ofspatial patterns and the sequence of sum values; (f) computing a focusindicator value based on the image, wherein the focus indicator valueindicates an extent to which the incident light stream is in focus atthe light modulation unit.
 23. A method for determining focus-relatedinformation for a compressive imaging (CI) device, the methodcomprising: (a) modulating an incident light stream with the sequence ofspatial patterns, thereby generating a modulated light stream, whereinsaid modulating is performed by a light modulation unit of the CIdevice; (b) generating an electrical signal representing intensity ofthe modulated light stream as a function of time, wherein the electricalsignal is generated by a light sensing device of the CI device; (c)acquiring samples of the electrical signal; (d) constructing an imageusing the sequence of spatial patterns and a subset of the acquiredsamples that corresponds to the sequence of spatial patterns; (e)computing focus information based on the image, wherein the focusinformation comprises information regarding at extent to which theincident light stream is in focus at the light modulation unit.
 24. Themethod of claim 23, wherein the number of spatial patterns in thesequence of spatial patterns is m, wherein the image is an n-pixel imagewith n less than the number N of light modulating elements in the lightmodulation unit, wherein m is less than n.
 25. The method of claim 23,further comprising: changing an optical distance between an opticalsubsystem of the CI device and the light modulation unit based on thefocus information.
 26. The method of claim 23, further comprising:changing a focus setting of an optical subsystem of the CI device basedon the focus information.
 27. A method for focusing a compressiveimaging (CI) device, the method comprising: (a) modulating an incidentlight stream with a sequence of spatial patterns to obtain a modulatedlight stream, wherein said modulating is performed by a light modulationunit of the CI device; (b) generating an electrical signal representingintensity of the modulated light stream as a function of time, whereinthe electrical signal is generated by a light sensing device of the CIdevice; (c) acquiring samples of the electrical signal, wherein theacquired samples include at least one sample for each of the spatialpatterns; (d) constructing an image using the sequence of spatialpatterns and the acquired samples; (e) displaying the image on adisplay; (f) changing a focus setting of the CI device based on userinput.
 28. The method of claim 27, wherein the focus setting of the CIdevice is an optical distance between an optical subsystem of the CIdevice and the light modulation unit.
 29. The method of claim 27,wherein the focus setting of the CI device is a focus setting of anoptical subsystem of the CI device.
 30. The method of claim 27, furthercomprising: performing (a) through (f) one or more additional times. 31.A method for focusing a compressive imaging (CI) device, the methodcomprising: (a) modulating an incident light stream with calibrationpatterns to obtain a modulated light stream, wherein said modulating isperformed by a light modulation unit of the CI device, wherein the lightmodulation unit includes an array of light modulating elements; (b) anarray of light sensing elements of the CI device receiving the modulatedlight stream, where the light sensing elements receive respectivespatial portions of the modulated light stream and generate respectiveelectrical signals, wherein each electrical signal represents intensityof the respective spatial portion as a function of time, wherein thelight sensing elements correspond to respective non-overlapping regionswithin the array of light modulating elements, wherein each of thecalibration patterns specifies that the light modulating elements insidea respective one of the regions are to be set to an off state andspecifies that at least a subset of the light modulating elementsoutside the respective region are to be set to an on state; (c) for eachof the calibration patterns, acquiring a respective group of samplesfrom the array of light sensing elements, wherein each of the samplegroups includes at least one sample of each of the electrical signals;(d) determining spillover values corresponding respectively to thecalibration patterns, wherein each of the spillover values is determinedbased on the sample group corresponding to the respective calibrationpattern, wherein each of the spillover values indicates an extent towhich modulated light from outside the respective region of the array oflight modulating elements reaches the light sensing elementcorresponding to the respective region. (e) computing a compositespillover value for the array of light sensing elements based on thespillover values.
 32. The method of claim 31, further comprising:directing an actuator to change an optical distance between the lightmodulation unit and the array of light sensing elements based on dataincluding the composite spillover value.
 33. The method of claim 31,further comprising repeatedly performing a set of operations thatincludes: changing an optical distance between the light modulation unitand the array of light sensing elements; and performing (a) through (e),wherein the set of operations is repeated until the composite spillovervalue becomes less than a threshold value.
 34. The method of claim 31,further comprising: directing an actuator to change an angle oforientation of the array of light modulating elements based on dataincluding the composite spillover value.
 35. The method of claim 31,further comprising repeatedly performing a set of operations thatincludes: changing an angle of orientation of the array of light sensingelements; and performing (a) through (e), wherein the set of operationsis repeated until the composite spillover value becomes less than athreshold value.
 36. The method of claim 31, wherein each of thecalibration patterns specifies that all the light modulating elementsoutside the respective region are to take the ON state.
 37. The methodof claim 31, further comprising: directing an actuator to change anoptical distance between the light modulation unit and the array oflight sensing elements according to a first displacement value;performing (a) through (e) a second time, to obtain an additionalcomposite spillover value; computing a second displacement value for theoptical distance between the light modulation unit and the array oflight sensing elements based on data including the composite spillovervalue, the additional composite spillover value and the firstdisplacement value.
 38. The method of claim 37 2.31, further comprising:directing the actuator to change the optical distance between the lightmodulation unit and the array of light sensing elements according to thesecond displacement value.
 39. The method of claim 31 2.27, furthercomprising: directing an actuator to change an angle of orientation ofthe array of light sensing elements according to a first displacementvalue; performing (a) through (e) a second time, to obtain an additionalcomposite spillover value; computing a second displacement value for theangular orientation based on data including the composite spillovervalue, the additional composite spillover value and the firstdisplacement value.
 40. The method of claim 39, further comprising:directing the actuator to change the angle of orientation of the arrayof light sensing elements according to the second displacement value.41. The method of claim 31, further comprising: directing an actuator tochange an optical distance between the light modulation unit and thearray of light sensing elements through a range of distances; performing(a) through (e) for each of a plurality of optical distances within therange in order to obtain a corresponding plurality of compositespillover values; determining an optimal value for the optical distancebased on an analysis of data including the plurality of compositespillover values.
 42. The method of claim 41, further comprising:directing the actuator to change the optical distance to the optimalvalue.
 43. The method of claim 31, further comprising: directing anactuator to change an orientation angle of the array of light sensingelements through a range of angles; performing (a) through (e) for eachof a plurality of angles within the range in order to obtain acorresponding plurality of composite spillover values; determining anoptimal value for the angle based on an analysis of data including theplurality of composite spillover values.
 44. The method of claim 43,further comprising: directing the actuator to change the angularorientation to the optimal value.
 45. A method for focusing acompressive imaging (CI) device, the method comprising: (a) modulatingan incident light stream with a calibration pattern to obtain amodulated light stream, wherein said modulating is performed by a lightmodulation unit of the CI device, wherein the light modulation unitincludes an array of light modulating elements; (b) an array of lightsensing elements of the CI device receiving the modulated light stream,where the light sensing elements receive respective spatial portions ofthe modulated light stream and generate respective electrical signals,wherein each electrical signal represents intensity of the respectivespatial portion as a function of time, wherein the light sensingelements correspond to respective non-overlapping regions within thearray of light modulating elements, wherein the calibration patternspecifies that the light modulating elements inside a first one of theregions are to be set to an off state and specifies that at least asubset of the light modulating elements outside the first region are tobe set to an on state, wherein the first region corresponds to a firstof the light sensing elements; (c) acquiring samples of the electricalsignal generated by the first light sensing element in response to saidmodulating the incident light stream with the calibration pattern,wherein each of the samples indicates an extent to which modulated lightfrom outside the first region of the array of light modulating elementsreaches the first light sensing element.
 46. A method for determiningfocus-related information for a compressive imaging (CI) device, themethod comprising: (a) modulating an incident light stream with asequence of spatial patterns to obtain a modulated light stream, whereinsaid modulating is performed by a light modulation unit of the CIdevice, wherein the light modulation unit includes an array of lightmodulating elements; (b) an array of light sensing elements of the CIdevice receiving the modulated light stream, wherein the light sensingelements receive respective spatial portions of the modulated lightstream and generate respective electrical signals, wherein each of theelectrical signals represents intensity of the respective spatialportion as a function of time, wherein the light sensing elementscorrespond to respective non-overlapping regions of the array of lightmodulating elements; (c) acquiring measurements from the array of lightsensing elements, wherein the measurements include sample sets thatcorrespond respectively to the light sensing elements, wherein eachsample set includes samples of the electrical signal generated by therespective light sensing element, with the samples including at leastone sample for each of the spatial patterns; (d) computing a focusindicator value based on two or more noise values for two or morerespective ones of the light sensing devices, wherein the focusindicator value indicates an extent to which the modulated light streamis in focus at the sensor array, wherein each of the two or more noisevalues is computed by: constructing a respective sub-image based on thesample set of the respective light sensing element and also on arestriction of the spatial patterns to the region of the array of lightmodulating elements corresponding to the respective light sensingelement; and computing an amount of noise present in the respectivesub-image.
 47. The method of claim 46, further comprising: determining adisplacement value for the array of light sensing elements based on dataincluding the focus indicator value; and directing the array of lightsensing elements to be translated according to the displacement value.48. The method of claim 46, further comprising: determining an angulardisplacement value for the array of light sensing elements based on dataincluding the focus indicator value; and directing the array of lightsensing elements to be rotated according to the angular displacementvalue.
 49. The method of claim 46, further comprising: changing anoptical distance between the light modulation unit and the array oflight sensing elements; and performing said changing and the operations(a) through (d) one or more times until the focus indicator value isoptimized.
 50. The method of claim 46, further comprising: changing anangular orientation of the array of light sensing elements; andperforming said changing and the operations (a) through (d) one or moretimes until the focus indicator value is optimized.
 51. A method fordetermining focus-related information for a compressive imaging (CI)device, the method comprising: (a) modulating an incident light streamwith a sequence of spatial patterns to obtain a modulated light stream,wherein said modulating is performed by a light modulation unit of theCI device, wherein the light modulation unit includes an array of lightmodulating elements; (b) an array of light sensing elements of the CIdevice receiving the modulated light stream, wherein the light sensingelements receive respective spatial portions of the modulated lightstream and generate respective electrical signals, wherein each of theelectrical signals represents intensity of the respective spatialportion as a function of time, wherein the light sensing elementscorrespond to respective non-overlapping regions of the array of lightmodulating elements; (c) acquiring a set of samples from a first of thelight sensing elements, wherein the sample set includes samples of theelectrical signal generated by the first light sensing element, with thesamples including at least one sample for each of the spatial patterns;(d) computing a focus indicator value for the first light sensingdevice, wherein the focus indicator value indicates an extent to whichthe modulated light stream is in focus at the sensor array, wherein thefocus indicator value is computed by: constructing a sub-image based onthe sample set and on a restriction of the spatial patterns to a theregion of the array of light modulating elements corresponding to thefirst light sensing element; and computing an amount of noise present inthe sub-image.
 52. A method for determining focus-related informationfor a compressive imaging (CI) device, the method comprising: (a)modulating an incident light stream with a sequence of spatial patternsto obtain a modulated light stream, wherein said modulating is performedby a light modulation unit of the CI device, wherein the lightmodulation unit includes an array of light modulating elements; (b) anarray of light sensing elements of the CI device receiving the modulatedlight stream, wherein the light sensing elements receive respectivespatial portions of the modulated light stream and generate respectiveelectrical signals, wherein each of the electrical signals representsintensity of the respective spatial portion as a function of time,wherein the light sensing elements correspond to respectivenon-overlapping regions on the array of light modulating elements; (c)acquiring measurements from the array of light sensing elements, whereinthe measurements include sample sets that correspond respectively to thelight sensing elements, wherein each sample set includes samples of theelectrical signal generated by the respective light sensing element,with the samples including at least one sample for each of the spatialpatterns; (d) computing a focus indicator value based on two or morespillover values for two or more respective ones of the light sensingdevices, wherein the focus indicator value indicates an extent to whichthe modulated light stream is in focus at the sensor array, wherein eachof the two or more spillover values is computed by: constructing arespective image based on the sample set of the respective light sensingelement and also on the spatial patterns; summing pixels in the imageoutside the region corresponding to the respective light sensingelement.
 53. A method for determining focus-related information for acompressive imaging (CI) device, the method comprising: (a) modulatingan incident light stream with a sequence of spatial patterns to obtain amodulated light stream, wherein said modulating is performed by a lightmodulation unit of the CI device, wherein the light modulation unitincludes an array of light modulating elements; (b) an array of lightsensing elements of the CI device receiving the modulated light stream,wherein the light sensing elements receive respective spatial portionsof the modulated light stream and generate respective electricalsignals, wherein each of the electrical signals represents intensity ofthe respective spatial portion as a function of time, wherein the lightsensing elements correspond to respective non-overlapping regions on thearray of light modulating elements; (c) acquiring measurements from thearray of light sensing elements, wherein the measurements include samplesets that correspond respectively to the light sensing elements, whereineach sample set includes samples of the electrical signal generated bythe respective light sensing element, with the samples including atleast one sample for each of the spatial patterns; (d) computing aspillover value for a first of the light sensing devices, wherein thespillover value indicates an extent to which the modulated light streamis in focus at the sensor array, wherein the spillover value is computedby: constructing an image based on the sample set corresponding to thefirst light sensing element and also on the spatial patterns; summingpixels in the image outside the region corresponding to the first lightsensing element.